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FOREWORD 


The  third  annual  Airborne  Weapons  Technology  Review  and  Training  Exposition  was 
held  14-16  January  1992  at  the  Princess  Resort  Hotel  in  San  Diego,  CA.  The  theme 
of  the  Exposition  was  weapons  training  for  the  year  2000  and  the  objective  was  to 
exchange  training  technologies  of  "today  and  tomorrow.”  The  Expo  was  sponsored  by 
Naval  Air  Systems  Command  (NAVAIRS7SC0M) ,  PMA  205  and  hosted  by  RDT&E  Division  of 
The  Naval  Command  Control  and  Ocean  Surveillance  Center  (NCCOSC). 

The  mission  of  the  Weapons  Training  2000  Program  is  to  design,  develop,  and 
execute  efficient,  cost-effective  weapons  training  programs  in  support  of  Fleet 
operational  readiness  and  maintenance  objectives  worldwide.  To  accomplish  this 
mission  NAVAIRSYSCOM  organized  a  Weapons  Training  Team  (WTT),  headed  by  Mr. 
William  Walker,  PMA  205-lH.  The  team  includes  several  Training  Systems  Managers 
(TSM)  within  PMA  205  and  the  field  activities.  This  team  has  developed  a 
Strategic  Plan  which  describes  the  approach  to  effectively  enhance  Naval  training 
in  the  face  of  escalating  demands  and  diminishing  resources. 

The  Weapons  Training  Team  was  organized  in  1987  and  every  year  the  team  gains  more 
support.  They  have  become  more  cohesive  in  1992  utilizing  lessons  learned  to 
identify  requirements  for  training,  staffing  and  Life  Cycle  Cost  (LCC).  A  sample 
of  their  goals  are:  interactive  multimedia,  computer-driven  Integrated  training 
systems,  deployable/distributive  training,  telecommunications,  and  possibly  most 
important  training  standardization. 

During  the  course  of  the  Expo  there  were  a  number  of  speakers  from  government, 
industry  and  academia,  that  presented  the  very  latest  in  Weapons  Training 
Technology.  Everything  from  a  training  system  specification  (in-process)  to  a 
visual/voice  activated,  hands-free  computer  (in  phase  I)  was  discussed.  Several 
Computer  Based  Training  (CBT)  programs  were  explained  Indepth,  stating  that  in 
their  development,  existing  hardware  and  software  were  utilized  and  integrated 
with  existing  Personal  Computers  (PCs). 

Training  on  actual  equipment  is  almost  a  thing  of  the  past  and  trainers 
(simulators)  are  becoming  very  expensive.  Therefore,  the  less  expensive  CBT 
technology  appears  to  provide  a  "better  way"  to  train.  Naval  and  civilian 
personnel  can  be  trained  more  quickly,  efficiently,  and  to  a  higher  level  of 
proficiency  with  CBT  and  other  new  technologies. 

The  opening  remarks  of  Captain  Walther  of  NCCOSC,  the  Introduction  by  Mr.  Walker, 
and  the  keynote  address  by  captain  Johannsen,  PMA  205,  all  dealt  with  the  latest 
reorganizations  and  reduction  in  force  and  funding.  All  DoD  agencies  must  prepare 
to  do  more  training  themselves  with  less  funding.  Business  practices  need  to 
change  also;  selecting  a  "best  value”  contract  over  the  lowest  bid  contract  can 
mean  a  savings  in  the  long  run. 

The  new  technology,  whether  in  the  development  stage  or  an  actual  program  in  use, 
is  truly  amazing.  Such  advances  as  the  touch  screen,  3-D  visual  and  audio, 
micro,  and  hands-free  computers  have  endless  applications.  In  some  cases  this  new 
technology  represents  teaming  with  competitors.  The  technology  is  present  today 
to  make  a  viable  paperless  technical  manual.  The  future  may  bring  technology  that 
has  built-in  (or  embedded)  training,  much  like  Built-in  Test  (BIT)  of  today. 
Training  must  be  the  number  one  priority  and  Total  Quality  Management/Total 
Quality  Leadership  (TQM/TQL)  was  stressed  to  all  participants. 


The  Expo  offered  additional  programniing  such  as  exhibits  on  display  for  those  who 
wished  to  have  some  hands-on  experience.  Many  of  the  CBT  devices  which  were 
discussed  could  be  seen  and  operated  at  the  exhibits.  Tours  were  also  offered  to 
those  who  wanted  to  visit  NAS  Miramar  or  NAS  North  Island  to  see  training 
facilities  and  associated  trainers  and  simulators.  Guest  speakers  during  the 
lunch  breaks  and  banquet  were  informative.  Panel  discussions  brought  out  some 
suggestions  that  will  be  staffed  and  considered  for  the  next  annual  Expo.  It  was 
observed  that  few  Navy  users  were  in  attendance,  however,  this  team/program  is 
still  very  new  and  additional  meetings  like  this  will  attract  more  Fleet  endline 
users. 

Master  of  Ceremony  duties  were  performed  by  Mr.  Joshua  Johnson  of  NCCOSC  who 
opened  the  meeting  and  kept  the  program  moving  along  to  keep  the  audience's 
attention.  A  phone  nvimber  for  the  Bulletin  Board  to  PMA  205-lH  was  given 
(800-472-4354).  PC  to  PC  correspondence  can  be  sent  through  this  number. 

The  world  of  technology  is  ever-changing  and,  when  you  relay  these  new 
technologies  to  the  training  community,  innovative  applications  are  developed  and 
become  the  key  to  success.  The  objectives  of  this  Expo  were  met  and  with 
continued  government  and  industry  support,  the  goals  set  forth  by  the  Weapons 
Training  2000  Team. 

Thanks  go  to  the  many  individuals,  and  their  supporting  organizations,  who  have 
given  so  generously  of  their  time  and  talents  to  make  the  third  annual  Airborne 
Weapons  Technology  Review  and  Training  Exposition  a  success. 


Joshua  Johnson 
Exhibition  Chairman 
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INTRODUCTION 


This  executive  overview  describes  the  concept  of  operations  for  the 
Tactical  Combat  Training  System  (TCTS) ,  which  will  support  Fleet  tactical 
training  and  readiness  evaluation.  This  overview,  which  follows  the  outline 
shown,  first  describes  the  purpose  of  the  system  and  then  offers  a  short 
discussion  of  the  major  shortfalls  in  current  operational  training  suppose 
that  are  intended  to  be  corrected  by  the  TCTS.  The  operational  performance 
requirements  am  then  reviewed  briefly  before  discussing  the  available 
technologies  and  instrumentation  functional  approach  that  make  it  feasible 
to  develop  a  TCTS  at  this  time.  To  further  clarify  the  concept  of 
operations,  a  graphic  example  is  presented  to  show  how  the  TCTS  would  be 
employed  operationally  in  support  of  a  Battle  Group  (BG)  training  exercise. 
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TCTS  CONCEPT  OF  OPERATIONS-EXECUTIVE  OVERVIEW 


OUTLINE 


PURPOSE 

SHORTFALLS  IN  CURRENT  OPERATIONAL  TRAINING  SUPPORT 
REQUIREMENTS  OVERVIEW 
INSTRUMENTATION  FUNCTIONAL  APPROACH 
EXAMPLE  BATTLE  GROUP  (BG)  TRAINING  SCENARIO 

SUMMARY 
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PURPOSE  OF  TOTS 


The  TCTS  is  required  to  satisfy  two  missions.  The  first  is  to  enhance 
all  aspects  of  current  and  projected  naval  combat  training,  and  the  second 
is  to  provide  the  data  collection  necessary  for  evaluating  fleet  exercises, 
developing  tactics,  and  carrying  out  Operational  Test  and  Evaluation  (OP&E) . 
Both  missions  are  based  upon  the  requirement  that  the  Fleet  exercise 
systems,  conduct  training,  and  evaluate  warfighting  effectiveness  at  sea. 
Therefore,  the  TCTS  addresses  unique  at-sea  training  requirements 
independently  from  classroom  and  pierside  training  requirements;  however, 
capabilities  that  are  being  developed  separately  to  support  coordinated 
multi-ship  pierside  training  will  be  considered  for  use  with  TCTS  at  sea. 

TCTS  instrumentation  will  be  designed  to  enhance  naval  combat  training 
from  single-platform  warfighting  performance  through  integration  of 
multi-platform  coordinated  combat  training  (surface,  subsurface,  and  air) 
and  to  include  integrated  Battle  Group/Force  multi-warfare  training.  To 
satisfy  this  purpose,  TCTS  instrumentation  will  be  designed  to  provide 
accurate,  realistic,  and  timely  feedback  of  exercise  activities  without 
artificially  constraining  the  exercise  participants  and  with  a  minimum 
impact  on  participant  support  requirements  (space  and  support  personnel). 
Also,  since  units  using  TCTS  may  be  deployed  to  remote  (potentially  hostile) 
locations  for  extended  periods,  the  system  will  be  designed  so  that  it  does 
not  interfere  with  the  operational  effectiveness  of  the  Fleet  and  so  that 
it  can  be  deployed  at  sea  for  long  periods  with  minimal  Fleet  logistical 
support  impact . 

The  instrumentation  system  developed  to  satisfy  Fleet  training 
requirements  will  also  provide  data  for  the  evaluation  of  Battle  Group 
readiness  and  tactical  effectiveness,  development  of  tactics,  and 
opsrational  test  and  evaluation  of  new  systems  prior  to  or  after  initial 
introduction  into  the  Fleet. 
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PURPOSE  OFTCTS 


•  ENHANCE  NAVAL  COMBAT  TRAINING 

“  SINGLE  PLATFORM  TOTAL  WARFIGHTING  TRAINING 

-  MULTI-PLATFORM  COORDINATED  COMBAT  TRAINING 

-  INTEGRATED  BATTLE  GROUP/FORCE  MULTI-WARFARE 

•  COLLECT  DATA  TO  SUPPORT  THE  ASSESSMENT  OF; 

-  BATTLE  GROUP  READINESS 

-  tactical  effectiveness 

-  TACTICS  DEVELOPMENT 

-  OTgE 


TRAINING 


5 


SHORTFALLS  IN  CURRENT  TACTICAL  TRAINING  SUPPORT 

The  Navy  trains  its  forces  in  all  warfare  areas  using  its  weapons  and 
sensor  systems  in  as  close  a  manner  as  possible  to  the  way  they 
would  be  employed  in  actual  combat.  The  Navy's  Tactical  Training  Range 
Roadmap  documents  various  factors  that  severely  limit  the  quality  and 
effectiveness  of  that  training; 

Most  at-sea  tactical  training  is  not  validated  since  there  is  insufficient 
instrumentation  available  to  support  an  objective  assessment  of  training 
objective  achievement.  Monitoring  of  exercises  is  both  limited  and 
qualitative,  and  the  lack  of  exercise  data  prevents  timely  effective 
training  feedback. 

Cost,  safety,  and  security  considerations  prevent  effective  training  in 
tactical  weapons  employment.  Many  sophisticated  weapons  are  so  expensive 
that  the  Fleet  cannot  afford  to  fire  them  except  as  required  for  weapon 
quality  monitoring  or  test  purposes.  Even  when  the  weapons  are  fired,  the 
safety  requirements  associated  with  having  actual  weapons  flyout,  whether 
carrying  explosive  or  dummy  warheads,  place  significant  constraints  on  the 
exercise  which  prevent  training  in  realistic  tactical  employment  of  the 
weapon.  Security  considerations  also  limit  realistic  training  employment  of 
some  weapon  systems. 

Most  at-sea  combat  training  is  conducted  without  realistic  threat 
representation.  Targets  are  generally  inadequate  both  in  characteristics 
and  numbers,  particularly  for  threat  air  platforms,  and  the  electronic 
warfare  environments  for  training  afloat  are  deficient  in  both  quantity  of 
threats  and  fidelity  of  the  threat  signatures.  The  cost  of  developing, 
procuring,  and  operating  target  and  threat  surrogates  at  sea  remains 
prohibitive. 

Normal  training  operating  areas  are  not  all  supported  by  collocated  fixed 
ranges.  Thus,  instrximented  training  capabilities  are  often  out  of  reach  of 
Fleet  units  requiring  training  support. 

Inability  to  score  strike  training  exercises  conducted  against  realistic 
(uninstrumented)  targets  of  opportunity  located  anywhere,  prevents  effective 
training  in  executing  attacks  directed  at  unfamiliar  targets. 

The  lack  of  multi-Warfare  training  instrumentation  that  is  deployable 
precludes  adequate  support  of  proficiency  maintenance  training  for  deployed 
forces.  Warfare,  skill  levels  achieved  at  one  point  of  the  training  cycle 
(e.g.,  CV  airwing  strike  warfare  skills  developed  on  the  Fallon  TACTS) 
become  degraded  during  deployment. 

Data  collection  to  support  Battle  Group/Force  evaluation  is  currently  labor 
intensive,  time-delayed,  and  has  no  adec[uate  common  reference  (ground 
truth) .  The  vast  database  of  Battle  Group  actions  and  event  outcomes  is 
collected  manually  through  system  operator  interviews  and  piecemeal 
compilation  of  existing  data  sources. 

As  a  result  of  these  major  shortfalls,  the  current  training  capability  does 
not  enable  the  Fleet  to  train  to  the  full  capabilities  of  current  weapons 
systems.  However,  available  technology  provides  means  to  eliminate  these 
shortfalls  through  the  development  of  tactical  training  instrumentation. 


6 


SHORTFALLS  IN  CURRENT  TACTICAL  TRAINING 

SUPPORT 


•  OBJECTIVE  ASSESSMENT  OF  MOST  TACTICAL  TRAINING 

-  MONITORING  CAPABILITY 

-  TIMEL  Y  INTERACTIVE  FEEDBA CK 

•  WEAPONS  EMPLOYMENT  TRAINING 

-  COST  CONSIDERATIONS 

-  SAFETY/SECURITY 

•  REALISTIC  THREAT  REPRESENTATION 

•  ACCESSIBILITY  OF  INSTRUMENTED  RANGES 

•  STRIKE  ASSESSMENT  AGAINST  TARGETS  OF  OPPORTUNITY  ANYWHERE 

•  PROFICIENCY  TRAINING  DURING  DEPLOYMENT 

•  DATA  COLLECTION  FOR  BATTLE  GROUP/FORCE  EVALUATION 
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TCT8  OPERATIONAL  PERFORMANCE  REQUIREMENTS — OVERVIEW 

The  TCTS  must  support  and  enhance  training  in  all  warfare  areas  (including 
those  pictured  on  this  chart)  by  providing  exercise  reconstruction  and 
timely  feedback  for  either  individual  or  combined  warfare  exercises.  The 
TCTS  also  is  required  to  provide  a  realistic  combat  environment,  including 
realistic  threats  and  exercise  scenarios.  TCTS  instrumentation  will  be 
required  to  support  any  combination  of  training  participants,  including 
land,  air,  sea,  and  undersea  platforms,  and  must  be  deployable  to  any 
geographical  area,  worldwide.  The  operational  performance  requirements  are 
discussed  in  more  detail  in  the  next  two  charts. 
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TCTS  OPERATIONAL  PERFORMANCE  REQUIREMENTS 

OVERVIEW 


TCTS  OPERATIONAL  PERFORMANCE  REQUIREMENTS 


The  TCTS  must  provide  the  tracking,  data  collection,  secure  data 
transmission,  processing,  and  display  necessary  to  conduct  and  assess 
tactical  training.  Data  interface  with  combat  systems  on  board  all  exercise 
participants  will  be  critical  to  (1)  collect  the  necessary  data  for 
performance  evaluation  and  exercise  monitoring,  and  (2)  inject  threat 
stimulation  data  into  the  combat  systems  during  the  exercise  to  increase 
tactical  realism  and  provide  immediate  performance  feedback. 

The  TCTS  must  provide  timely  feedback  of  training  results  to  the 
exercise  participants.  This  requires  both  real-time  scoring  sufficient  to 
maintain  battle  realism  and  kill  removal  during  interactive  engagements  and 
immediate  post-operation  reconstruction  and  debrief  to  all  participants  to 
get  the  greatest  training  value  from  each  exercise. 

Discussions  with  the  Fleet  also  indicate  a  strong  desire  for  the  TCTS 
to  be  able  to  autonomously  suppose  training  exercises  involving  scenarios 
up  to  Battle-Group  size  without  needing  shore-based  assets  or  direct 
assistance. 

The  TCTS  must  be  designed  so  that  it  will  be  deployable  at  sea  in 
remote  operational  areas  for  long  periods  of  time  to  provide  proficiency 
training  throughout  the  entire  deployed  period.  The  core  instrumentation 
(hardware  and  software)  must  be  highly  portable  and  compact  so  it  can  be 
installed  within  available  space (s)  on  any  flag-configured  ship.  This 
portability  will  allow  any  Battle  Group  or  Amphibious  Ready  Group  to  be 
equipped  with  TCTS  to  support  all  phases  of  at-sea  exercises.  The  core 
instrumentation  should  also  be  configurable  for  installation  at  a  fixed 
shore ^  site.  The  TCTS  should  be  designed  to  facilitate  evolution  to  an 
organic  system  that  can  be  embedded  on  every  operational  Fleet  unit  in  the 
more  distant  future. 
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TCTS  OPERATIONAL  PERFORMANCE  REQUIREMENTS 


•  PROVIDE  TRACKING,  DATA  COLLECTION,  SECURE  DATA  TRANSMISSION, 

PROCESSING  &  DISPLAY  NECESSARY  TO  CONDUCT  &  ASSESS  TACTICAL  TRAINING 


•  FINISH  TIMELY  FEEDBACK  OF  TRAINING  RESULTS  TO  PARTICIPANTS 

-  REAL-TIME  SCORING  &  KILL  REMOVAL 

-  IMMEDIATE  POST-OP  REPLAY/DEBRIEF 


•  PROVIDE  AUTONOMOUS  TRAINING  CAPABILITY  FOR  EXERCISES  UP  TO  BATTLE 
GROUP  SIZE 


•  BE  DEPLOYABLE  TO  SUPPORT  WORLDWIDE  NAVY  TRAINING  EXERCISES 

-  ALL  TYPES  OF  TACTICAL  TRAINING,  OVER  LAND  AND/OR  SEA 

-  CAPABLE  OF  LONG-DURATION  AT-SEA  DEPLOYMENT  CYCLES 

-  HIGHLY  PORTABLE  &  COMPACT 

-  EVOLUTION  TO  AN  EMBEDDED  SYSTEM  ON  ALL  NA  VY  SHIPS  &  AIRCRAFT 
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TCT8  OPERATIONAL  PERFORMANCE  REQUIREMENTS  (Concluded) 


The  TCTS  must  be  able  to  accommodate  a  mix  of  each  possible  threat 
representation  in  any  desired  combination  to  support  a  full  range  of  at- 
sea  training  and  BG  evaluation  exercises,  including: 

*  Manned  or  unmanned  platforms  used  to  represent  threat  systems. 

*  Computer-generated  simulated/stimulated  pseudotargets  and 
electromagnetic (EM)  signals  introduced  into  the  operational 
sensors/displays  of  participant  platforms  to  improve  training  realism 
(by  increasing  threat  density) . 

*  EM  signal  stimulation  must  be  implemented  so  that  it  does  not  preclude 
reception  of  real-world  signals  of  opportunity  (e.g.,  radiating  threat 
emitters  and  slcin  return  from  actual  targets)  . 

The  TCTS  must  interface  and  exchange  data  with  other  ranges  to  provide 
interoperable  capabilities.  TCTS-equipped  aircraft  -participants  launched 
from  carriers  operating  offshore  must  be  able  to  fly  strike  training 
missions  to  an  TACTS/ ACMI -type  ranges,  including  those  located  at  NAS  Fallon 
and  Nellis  AFB,  Nevada.  Aircraft  participating  in  at-sea  exercises  can  then 
take  advantage  of  the  fixed  range  capabilities,  including  scorable  targets 
and  Shrike,  HARM,  and  EW  training.  The  TCTS  must  also  be  able  to  exchange 
data  with  Fleet  ranges  (e.g.,  SCORE,  AFWTF,  and  PMRF)  whenever  TCTS- 
equipped  Fleet  units  are  operating  in  their  vicinity.  TCTS  will  then  be  able 
to  incorporate  data  collected  by  the  fixed  range  systems  (e.g.  surveillance, 
EW  training,  and  underwater  tracking  data) ,  and  the  fixed  ranges  will  be 
able  to  extend  their  surface  and  in-air  instrumentation  coverage  using  data 
from  the  TCTS. 

TCTS  instrumentation  must  be  designed  so  that  it  does  not  constrain  or 
limit  the  operational  performance  of  training  participants  (operational 
ships  and  aircraft  involved  in  training  exercises) .  This  will  allow  Fleet 
units  deployed  in  remote  areas  to  use  the  TCTS  for  day-to-day  training  and 
at  the  same  time  maintain  full  operational  readiness.  The  main  impacts  of 
this  requirement  will  be  the  need  to  ensure  that  training  data  are 
transmitted  independently  of  Fleet  tactical  communications  functions  and 
that  the  approach  used  for  tracking  exercise  participant  positions  does  not 
constrain  the  tactical  positioning  of  participating  units  as  current  systems 
do. 
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TCTS  OPERATIONAL  PERFORMANCE  REQUIREMENTS 


•  ENHANCE  THREAT  REALISM  BY  SUPPORTING  ANY  MIX  OF  THREAT  REPRESENTATIONS 

-  MANNED/UNMANNED  THREAT  PLATFORMS 

-  SIMULATED  THREAT  PLATFORMS  &  EMITTERS  (e.g..  COMPUTER-GENERATED  STIMULATION  OF 
PARTICIPANT  SENSOR  SYSTEMS) 

-  RADIATING  THREAT  EMITTERS 

•  INTERFACE  &  EXCHANGE  DATA  WITH  OTHER  RANGES  (INTEROPERABILITY) 

-  TACTS/ACMI  TYPE  RANGES  (e.g..  FALLON) 

-  UNDER  WA  TER  RANGES  A  T  SCORE.  AFWTF.  &  PMRF 


•  INSTRUMENTATION  MUST  NOT  CONSTRAIN  OPERATIONAL  CAPABILITIES  &  TACTICS  OF 
TRAINING  EXERCISE  PARTICIPANTS 

-  OPERATIONAL  CAPABILITY  OF  PARTICIPANTS  NOT  DEGRADED  BY  INSTRUMENTATION 

-  TRAINING  DATA  TRANSMISSION  INDEPENDENT  OF  FLEET  TACTICAL  FUNCTIONS 

-  TACTICAL  SETUPS  NOT  CONSTRAINED  BY  INSTRUMENTATION 
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INSTRUMENTATION  FUNCTIONAL  AFPROACH->OVERVIEW 


The  development  of  TCTS  is  expected  to  use  some  technology  previously  proven 
by  other  training  instrumentation  systems.  Additional  development  efforts 
in  certain  technical  areas  are  complementary  for  TCTS  and  other  programs; 
it  is  expected  that  these  efforts  will  be  pursued  in  a  coordinated  manner. 

The  Tactical  Aircrew  Combat  Training  System  (TACTS)  and  Mobile  Sea  Range 
(MSR)  programs  have  accomplished  substantial  development  in  areas  directly 
transferrable  to  TCTS: 

*  Aircraft  and  ship  interfaces — hardware  and  software  needed  to 
extract  platform  weapon  and  sensor  system  data,  and  to  allow  access  for 
injecting  signals  into  those  systems. 

*  Instrumentation  datalink — technology  used  for  the  MSR  may  have 
application  to  TCTS. 

*  Weapons  simulations — computer-generated  weapons  flyout,  threat 
representation  models. 

*  Sensor  stimulation — investigation  of  embedded  sensor  system 
simulators  and  stimulation  techniques. 

*  Datalink  encryption — COMSEC  device  and  techniques  developed  for 
TACTS  and  MSR  are  directly  applicable  to  the  datalink  security 
requirements  of  TCTS. 

The  Battle  Force  Tactical  Training  (BFTT)  program  also  requires  ship 
interfaces  and  ship  sensor  system  stimulation  technology.  Development  of 
common  interfaces  for  BFTT  and  TCTS  is  a  program  goal. 

Development  of  upgraded  airborne  instrumentation  subsystem  (UPOD) 
common  participant  instrumentation  equipment  (in  pod  and  internal 
configurations)  for  general  training  instrumentation  applications  (including 
TACTS/ACMI  and  MSR)  is  an  important  complementary  program  that  will 
contribute  to  TCTS  goals. 

The  Global  Positioning  System  (GPS),  which  will  provide  accurate 
psi^ticipant  position  data  and  timing  independent  of  fixed  sites,  has  the 
demonstrated  potential  to  provide  the  TCTS  tracking  base. 

Available  state-of-the-art  technology  will  provide  many  advanced 
capabilities  and  dramatic  improvements  in  performance/ size  ratios  of 
products  that  can  be  incorporated  into  TCTS  hardware  subsystems. 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 

APPLY  PROVEN  &  AVAILABLE  TECHNOLOGY  TO  OVERCOME  SHORTFALLS 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 


The  instrumentation  functional  approach  for  the  TCTS  builds  upon  proven 
training  systems  through  the  addition  of  currently  available 
state-of-the-art  technology.  The  principal  functions  required  of  the  TCTS 
are: 

Position  location,  tracking,  and  time  synchronization  of  all  exercise 
participants 

Interface (s)  with  exercise  participants  to  collect/ inject  warfare  systems 
status  and  event  data 

Data  transmission  capacity  for  necessary  exercise  data  collection, 
interaction  (e.g.,  kill  removal),  and  timely  training  feedback 

Coordinated/distributed  data  processing  capability  to  support  realistic 
interactive  exercises 

Display  and  debrief  capability  to  conduct  and  evaluate  multiwarfare 
exercises. 

The  TCTS  functional  approach  in  each  of  these  areas  is  addressed  in 
subsequent  charts. 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 

REQUIREMENTS 


•  POSITION  LOCATION.  TRACKING.  &  TIME  SYNCHRONIZATION  OF  ALL 
EXERCISE  PARTICIPANTS 

•  INTERFACE(s)  WITH  EXERCISE  PARTICIPANTS  TO  COLLECT/iNJEa 
WARFARE  SYSTEMS  STATUS  &  EVENT  DATA 

•  DATA  TRANSMISSION  CAPACITY  FOR  NECESSARY  EXERCISE  DATA 
COLLECTION.  INTERACTION  (e  g..  KILL  REMOVAL)  &  TIMELY  TRAINING 
FEEDBACK 

•  COORDINATED/DISTRIBUTED  DATA  PROCESSING  CAPABILITY  TO 
SUPPORT  REALISTIC  INTERACTIVE  EXERCISES 


•  DISPLAY  &  DEBRIEF  CAPABILITY  TO  CONDUCT  &  EVALUATE 
MULTIWARFARE  EXERCISES 
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POSITION  LOCATION,  TRACKING,  AND  TIME  SYNCHRONIZATION 


Position  location,  tracking,  and  time  synchronization  will  be 
accomplished  using  an  integrated  approach  employing  Global  Positioning 
System  (GPS)  data  from  all  participants  supplemented  with  attitude,  angular 
rates,  and  acceleration  data  obtained  from  inertial  units  to  accomplish 
state-vector  tracking  of  those  aircraft  engaged  in  high-dynamics 
maneuvering. 

The  GPS  and  inertial  data  will  be  acquired  from  participant  platform 
organic  GPS  and  inertial  units  whenever  feasible.  For  participants  not 
equipped  with  organic  GPS  receivers  or  inertial  units,  GPS  and/or  inertial 
equipment  will  be  provided  as  part  of  the  platform's  TCTS  instrumentation. 

Time  synchronization  will  be  accomplished  in  each  participant's  TCTS 
instrumentation  unit  using  the  accurate  timing  data  provided  by  GPS.  Each 
participant's  state-vector  data  will  be  time-tagged  by  the  TCTS 
instrumentation  unit  and  transmitted  to  the  TCTS  core  instrumentation  in 
near-real  time  via  the  TCTS  instrumentation  datalink. 

Submarine  position  data  will  be  collected  from  one  of  two  sources: 
either  post-event  from  the  onboard  tactical  inertial  navigation  system;  or 
in  near-real  time  from  fixed  or  portable  underwater  range  acoustic  tracking 
systems  if  exercise  location  permits. 


INSTRUMENTATION  FUNCTIONAL  APPROACH 

POSITION  LOCATION,  TRACKING,  &  TIME  SYNC 


track  all  surface  &  AIR  EXERCISE  PARTICIPANTS  USING  POSITION/VELOCITY  DATA 
OBTAINED  FROM  GLOBAL  POSITIONING  SYSTEM  (GPS)  UNITS  ON  EACH  PARTICIPANT 

SUPPLEMENT  GPS  DATA  WITH  STATE-VECTOR  DATA  (ATTITUDE,  ANGULAR  RATES.  & 
ACCELERATION)  OBTAINED  FROM  INERTIAL  UNITS  ON  SELECTED  HIGH-DYNAMICS  AIRCRAFT 

PARTICIPANTS 

ACQUIRE  GPS  &  INERTIAL  DATA  FROM  PARTICIPANT  PLATFORMS'  ORGANIC  TACTICAL  GPS 
&  INERTIAL  UNITS  WHENEVER  FEASIBLE  (USUALLY  VIA  INTERFACE  WITH  ONBOARD  DATA  BUSES) 

-  PROVIDE  GPS/INERTIAL  UNITS  AS  PART  OF  TOTS  INSTRUMENTATION  (PODS  OR  INTERNAL  UNITS) 
FOR  PARTICIPANTS  NOT  EOUIPPED  WITH  ORGANIC  TACTICAL  GPS/INERTIAL  UNITS 


SYNCHRONIZE  EACH  PARTICIPANT  WITH  GPS  TIME  BASE 


•  TRANSMIT  TIME-TAGGED  GPS/INERTIAL  DATA  FROM  PARTICIPANTS  TO  TCTS  CORE 
INSTRUMENTATION  IN  NEAR-REAL  TIME  VIA  TCTS  INSTRUMENTATION  DATALINK 


•  RECEIVE  POST-EVENT  TRACK  DATA  OBTAINED  FROM  SUBMARINES'  ONBOARD  TACTICAL 
INERTIAL  NAVIGATION  SYSTEMS  &  INTEGRATE  INTO  TCTS  CORE  INSTRUMENTATION  IN 
NON-REAL  TIME 

-  INCORPORATE  NEAR-REAL-TIME  TRACK  DATA  FROM  FIXED  OR  PORTABLE  UNDERWATER  RANG 
WHEN  TCTS  OPERATIONS  ARE  CONDUCTED  IN  THEIR  VICINITY 


INTERFACES  WITH  EXERCISE  PARTICIPANTS 


Interfaces  with  exercise  participants  are  required  to  capture  the 
warfare  systems  status  and  event  data  necessary  for  exercise  evaluation  and 
management  and  to  provide  for  data  injection  to  simulate  platform  tactical 
systems.  TCTS  will  take  advantage  of  existing  aircraft  and  ship  interfaces 
and  those  under  development  for  other  programs  including:  aircraft  and  ship 
interfaces  developed  for  the  TACTS/ACMI  and  MSR  programs,  pierside  ship 
combat  system  training  interfaces,  and  shipboard  embedded  training  system 
interfaces.  A  continuing  program  to  introduce  platform  training  interface 
standards  to  guide  design  of  new  ship  and  aircraft  systems  is  recommended. 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 

INTERFACES  WITH  EXERCISE  PARTICIPANTS 


•  MAKE  MAXIMUM  USE  OF  EXISTING  DATA  CAPTURE/DATA  INJECTION  INTERFACES 
&  THOSE  UNDER  DEVELOPMENT; 

-  TACTS/ACMI 

-  MSR 

PIERSIDE  SHIP  COMBAT  SYSTEM  TRAINERS 

-  STIMULATORS/TRAINERS  EMBEDDED  IN  COMBAT  SYSTEMS 

•  DEVELOP  NEW  INTERFACES  AS  REQUIRED 

•  DEVELOP  RECOMMENDED  PLATFORM  TRAINING  INTERFACE  STANDARDS  TO  GUIDE 
DESIGN  OF  NEW  SHIP  &  AIRCRAFT  SYSTEMS 

-  COORDINATED  WITH  OTHER  TRAINING  INTERFACE  USER  REQUIREMENTS 

-  EVOLUTION  TO  NAVY-WIDE  ANDIOR  DoD  TRAINING  INTERFACE  STANDARDS 
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TCTS  INTERFACE  WITH  EXISTING  SHIPBOARD-EMBEDDED  TRAINING  SYSTEMS 


Embedded  training  systems  will  be  in  place  as  organic  equipment  on 
many  combatant  ship  classes  during  the  1990s.  These  systems  include  the 
AN/USQ-93(V)  Radar  Environment  Simulator  System  (RESS) ,  the  Advanced  Combat 
Direction  System  (ACDS)  Shipboard  Training  and  Assessment  Group  (STRAG)  ,  the 
Aegis  Combat  Training  System  (ACTS) ,  the  AN/SSQ-91(V)  Combat  Simulation  Test 
System  (CSTS) ,  and  the  AN/SQQ-89T(V)  On-Board  Trainer  (OBI).  Each  of  these 
embedded  training  systems  will  interface  with  various  shipboard  tactical 
systems  installed  on  specific  ship  classes.  These  training  interfaces  will 
provide  for  stimulation  of  tactical  sensor  and  combat  systems  by  the 
embedded  trainer  and  for  monitoring/collection  of  data  from  the  tactical 
systems . 

TCTS  will  make  use  of  these  available  stimulation  and  data  monitoring 
capabilities  by  interfacing  with  a  ship's  embedded  training  system  rather 
than  with  the  multiple  tactical  systems  individually.  TCTS  will  coordinate 
the  activities  of  the  simulators  in  real  time  so  that  the  shipboard  sensor 
systems  are  stimulated  appropriately  in  consonance  with  the  dynamic  TCTS 
training  scenario.  The  TCTS  instrumentation  unit  will  consolidate  data 
collected  from  shipboard  tactical  systems  via  the  embedded  training  system 
and  transfer  data  to  the  TCTS  core  instrumentation  (and  other  participants 
as  necessary)  via  the  TCTS  instrumentation  datalink. 
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TCTS  WILL  INTERFACE  WITH  EXISTING  SHIPBOARD 
EMBEDDED  TRAINING  SYSTEMS 


EMBEDDED  TRAINING  SYSTEM 

1 

SENSOR  SYSTEMS  THAT  ARE 
STIMULATED 

APPLICABLE  SHIPS 

AN/USQ-93(V)  RADAR  ENVIRONMENT 
SIMULATOR  SYSTEM  (RESS) 

TRACKING  &  SURVEILLANCE 
RADAR/IFF  (RF),  EW  (GROWTH) 

NEW  THREAT  UPGRADE  SHIPS: 

ALL  NON-AEGIS  CRUISERS  &  KIDD 
CLASS  DESTROYERS 

(32  SHIPS) 

ADVANCED  COMBAT  DIRECTION 
SYSTEM/SHIPBOARD  TRAINING  & 
ASSESSMENT  GROUP  (ACDS/STRAG) 

BLOCK  1  PLANS:  TRACKING  & 
SURVEILLANCE  RADAR/IFF, 

AS\A/,  EVI^  (USING  RESS  &  SM-441) 

VARIOUS  CV/CVN,  CG/CGN,  & 
DD/DDG  (POTENTIALLY  OVER  150 

SHIPS  BY  LATE  1990s) 

AEGIS  COMBAT  TRAINING  SYSTEM 
(ACTS) 

AN/SPY-1  RADAR/IFF  (ASW  WHEN 
INTEGRATED  WITH  OBT) 

ALL  AEGIS  CRUISERS  & 
DESTROYERS  (30  SHIPS  BY 

MID-1990S) 

AN/SSQ-91(V),  COMBAT  SIMULATION 
TEST  SYSTEM  (CSTS) 

TRACKING  &  SURVEILLANCE 
RADAR/IFF  USING  RF  &  VIDEO 

MIX  (CSTS  STIMULATORS  &  SM-441) 

LHD-1  CLASS  (6  SHIPS) 

AN/SQQ-89T(V),  ON-BOARD  TRAINER 
(OBT) 

ASW,  DIGITAL  STIMULATION  OF 
EW  (ALQ-142  A  SLQ-32  ESM) 

AEGIS  CRUISERS  &  DESTROYERS; 
KIDD  &  SPRUANCE  CLASS 
DESTROYERS;  PERRY  CLASS 
FRIGATES  (APPROX.  120  SHIPS) 
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DATA  TRANSMISSION 


Exercise  data  transmission  requirements  will  be  satisfied  by  providing 
secure  instrumentation  datalink  capabilities  between  the  TCTS  core  (master 
processor)  instrumentation  and  all  participants  and  by  employing  long-range 
wideband  datalink  capabilities  to  interconnect  the  core,  shore  support 
facility(s),  fixed  ranges,  and  remote  debriefing  sites  afloat  and  ashore. 

A  real-time  RF  datalink  will  allow  basic  instrumentation  data  exchange 
between  the  core  instrumentation  and  all  surface  and  airborne  participants. 
The  datalink  must  be  independent  of  Fleet  tactical  functions  to  avoid 
loading  tactical  links  with  TCTS  instrumentation  traffic.  Frequency 
selection  should  avoid  bands  that  might  be  jammed  during  exercises. 

This  datalink  must  be  encrypted  for  data  security  and  designed  to  allow 
interparticipant  communications  to  be  conducted  between  nearby  participants 
during  detached  operations  when  connectivity  with  the  core  instrumentation 
is  interrupted. 

Two  approaches  are  currently  under  consideration  for  submarine 
post-event  data  transmission:  data  transmission  from  communications  depth 
(e.g.,  via  satellite  relay  channel),  and  use  of  a  SLOT-buoy  type  device 
(e.g.,  AN/BRT-6  SATCOM  buoy)  that,  after  being  loaded  with  data  and  released 
from  operating  depth,  will  rise  to  the  surface  and  transmit  the  data  via  the 
SSIXS  channel. 

Satellite  links  (either  leased  or  government)  will  permit  data 
transmission  between  the  core  instrumentation  and  distant  facilities  to 
support  large  multiforce  exercises,  relay  readiness  evaluation  data  for 
analysis,  interface  with  fixed  ranges,  and  support  exercise  debriefs  at 
remote  sites. 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 

DATA  TRANSMISSION 


•  RF  DATALINK  BETWEEN  CORE  INSTRUMENTATION  &  AIR/SURFACE  PARTICIPANTS 

-  INDEPENDENT  OF  TACTICAL  FUNCTIONS 

•  AVOID  LOADING  TACTICAL  LINKS  WITH  INSTRUMENTATION  TRAFFIC 

•  AVOID  JAMMING  TARGETED  AT  TACTICAL  COMM,  RADAR,  &  DATALINKS 

-  END-TO-END  ENCRYPTION  FOR  DATA  SECURITY 

-  DA  TA  TRANSMISSION  IN  NEAR-REAL  TIME  FOR  AIR  &  SURFACE  PARTICIPANTS 

•  POST-EVENT  DATA  TRANSMISSION  FOR  SUBSURFACE  PARTICIPANTS 

•  SATELLITE  LINKS  BETWEEN  CORE  INSTRUMENTATION  &  SHORE  SUPPORT 
FACILITY(s)/FIXED  RANGES/REMOTE  DEBRIEF  SITES  (AFLOAT  &  ASHORE) 
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COORDINATED/DISTRIBUTED  DATA  PROCESSING 


TCTS  will  employ  a  distributed  architecture  in  which  each  participant's 
onboard  TCTS  instrumentation  package  must  perform  extensive  local  data 
processing  functions  that  are  coordinated  with  the  overall  training 
scenario.  Current  state-of-the-art  microprocessor  technology  can  provide 
the  processing  throughput  necessary  to  execute  these  functions  locally  and 
to  coordinate  with  other  participants  and  the  TCTS  core  instrumentation. 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 

COORDINATED/DISTRIBUTED  DATA  PROCESSING 


•  DISTRIBUTED  PARTICIPANT  INSTRUMENTATION  UNIT  HARDWARE/SOFTWARE 
WITH  CAPACITY  TO: 

-  MONITOR  CURRENT  OWNSHIP  POSITION.  HEADING,  etc. 

-  MONITOR  POSITION  REPORTS  FROM  OTHER  NEARBY  (REAL)  PARTICIPANTS 

-  SIMULA  TE  LOCAL  THREA  T  ENVIRONMENT  (PER  THREA T  SCENARIO  DA TABASE)  & 
GENERATE  STIMULATION  PARAMETERS  FOR  INJECTION  INTO  PARTICIPANT 
SENSOR  &  WEAPON  SYSTEMS 

-  CORRELATE  PARTICIPANTS  TARGET  TRACKS  WITH  REFERENCE  TRACK 
INFORMATION  (SIMULATED  OR  REAL) 

-  PERFORM  WEAPON/COUNTERMEASURES  EMPLOYMENT  EFFECTS 
ASSESSMENT  (e.g.,  MISSILE  FLYOUT  SIMULATION)  &  PROVIDE  FEEDBACK  TO 
PARTICIPANTS 

-  ARCHIVE  &  DOWNLINK  (TO  CORE  INSTRUMENTA TION)  CRITICAL  INFORMA TION 
TO  SUPPORT  DEBRIEF  &  PERFORMANCE  ASSESSMENT 


CENTRALIZED  DATA  PROCESSING 


TCTS  instrumentation  installed  on  ships  will  incorporate  the  core 
instrumentation  functions  needed  to  coordinate  and  monitor  database 
generation  and  distribution,  exercise  execution,  data  correlation,  debrief 
database  generation  and  dissemination,  and  data  interchange  with  remote 
facilities  afloat  and  ashore.  One  ship  will  be  designated  to  perform  the 
core  instrumentation  functions  for  each  specific  exercise.  It  also  will 
be  possible  to  configure  fixed  shore  sites  to  perform  the  core 
instrumentation  functions  for  exercises  in  their  vicinity. 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 

CENTRALIZED  DATA  PROCESSING 


•  SHIPS  CONFIGURED  TO  SUPPORT  CORE  INSTRUMENTATION  FUNCTIONS: 

-  GENERA  TE  SIMULA  TED  THREA  T  SCENARIO  DA T ABASE  &  DISTRIBUTE  SUBSETS  TO 
EACH  PARTICIPANT  PRIOR  TO  EXERCISE  START 

-  MODIFY  THREA  T  SCENARIO  DA  T ABASE  DURING  EXERCISE  (in  response  to  opera  tor 
ENTRIES)  &  DISTRIBUTE  CHANGES  TO  AFFECTED  PARTICIPANTS 

-  MAINTAIN  POSITION  &  MOTION  STA  TE-VECTOR  LOG  ON  ALL  PARTICIPANTS 

-  CORRELA  TE  POSITION  DA  TA  WITH  TARGET  TRACKING,  STA  TUS.  <S  EVENT  DA TA 
COLLECTED  FROM  PARTICIPANT  SENSORS  &  WEAPON  SYSTEMS 

-  GENERA  TE  CONSOLIDA  TED  DEBRIEFING  DA TABASE 

-  PRE-PROCESS  PERFORMANCE  ASSESSMENT  INFORMATION  FOR  TRANSMISSION  TO 
SHORE  FACILITY(s) 


•  FOR  EACH  SPECIFIC  EXERCISE,  ONE  SHIP  WILL  BE  DESIGNATED  TO  PERFORM  THE 
CORE  INSTRUMENTATION  FUNCTIONS 


•  FIXED  SHORE  SITES  CAN  ALSO  BE  CONFIGURED  TO  PERFORM  CORE 
INSTRUMENTATION  ROLE 
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DISPLAY  AKD  DEBRIEF  CAPABILITY 


Display  and  debrief  are  required  to  take  full  advantage  of  the 
extensive  training  and  assessment  capabilities  of  the  TCTS.  Display 
capabilities  are  necessary  for  performing  shipboard  core  instrumentation 
and  operating/debriefing  functions  and  for  interfacing  with  display 
equipment  at  fixed  shore  sites  for  debriefing  and  exercise  assessment  and 
evaluation. 

The  latest  NDI  technology  is  considered  satisfactory  for  development 
of  the  portable  operating  and  debriefing  stations  needed  aboard  ships  for 
training  exercise  coordination  and  debriefing.  These  displays  will  be  highly 
portable,  rugged,  and  compact,  suitable  for  the  most  flexible  installation 
in  very  restricted  shipboard  space (s) .  Interfaces  also  will  be  provided  to 
permit  the  use  of  compatible  embedded  shipboard  operational  displays  for 
debrief  of  TCTS-supported  exercises  and  for  displaying  data  received  from 
fixed  ranges  at  shipboard  operating/debriefing  stations. 

Interfaces  for  data  exchange  with  fixed-range  and  shore  support 
facility  display  systems  will  be  developed  to  provide  for  remote  debriefing 
and  optional  shore-based  coordination  support  for  large-scale  exercises,  and 
to  provide  assessment  data  in  support  of  post-exercise  Battle  Group 
evaluation  and  assessment  by  Fleet  analysts. 
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INSTRUMENTATION  FUNCTIONAL  APPROACH 

DISPLAY  &  DEBRIEF  CAPABILITY 


•  USE  LATEST  NDl  DISPLAY  TECHNOLOGY  (e.g.,  TAC-Ill)  TO  PROVIDE  PORTABLE 
OPERATING/DEBRIEFING  STATIONS(s)  THAT  CAN  BE  FLEXIBLY  CONFIGURED  & 
DISTRIBUTED  IN  SHIPBOARD  OPERATING  SPACES 

-  MULTIPLE  STATIONS  FOR  CORE  INSTRUMENTATION  EXERCISE  SETUP.  MONITORING. 
CONTROL.  &  DEBRIEF 

-  OPERATING/DEBRIEFING  STATION(s)  FOR  OTHER  SHIP  PARTICIPANTS 

•  SHIPBOARD  INTERFACES  TO  ALLOW  USE  OF  OPERATIONAL  DISPLAYS 
(e.g.,  UYQ-21)  FOR  POST-EXERCISE  DEBRIEFING 

•  CAPABILITY  TO  DISPLAY  DATA  RECEIVED  FROM  FIXED  RANGES  AT  SHIPBOARD 
OPERATING/DEBRIEFING  STATIONS 


•  REMOTE  DEBRIEFING  AT  EXISTING  SHORE  FACILITIES  (e.g., TACTS,  FIXED  RANGES) 
THROUGH  TCTS  INTERFACES  TO  EXISTING  DISPLAY  SYSTEM(s) 

•  INTERFACES  WITH  SHORE  SUPPORT  FACILITY  DISPLAYS  FOR  OPTIONAL  EXERCISE 
COORDINATION  SUPPORT  &  FOR  POST-EXERCISE  BATTLE  GROUP  EVALUATION  & 
ASSESSMENT  SUPPORT 
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TCT8  TOP-LEVEL  NOTIONAL  DESIGN 

The  top-level  TCTS  notional  design  illustrated  below  indicates  the  basic 
system  elements  and  interfaces  necessary  to  satisfy  the  TCTS  concept  of 
operations.  A  few  basic,  elements  form  the  design — participant 
instrumentation  for  ship  and  aircraft  installation,  the  core  instrumentation 
functions  to  be  exercised  by  a  participating  ship  or  at  a  fixed  site, 
interfaces  with  fixed  ranges  for  two-way  data  interchange,  optional 
interfaces  and  debriefing  equipment  for  installation  at  shore-based  training 
simulator  facilities,  optional  debriefing  equipment  (installable  at  either 
ship  or  shore  sites  for  remote  debriefing) ,  and  interfaces  with  shore 
support  facilities.  All  of  these  elements  are  readily  achievable  using 
current  technology.  ^ 

*  P®r.^icipant  instrumentation  consists  of  a  GPS  receiver  to  determine 
position  and  t^e,  interfaces  and  monitoring  equipment  to  collect  the 
sensor/weapons/ cr  data  during  each  exercise,  and  stimulation  interfaces  and 
processing  needed  to  exercise  participant  equipment  as  directed  by  the  core 
system.  A  local  processor  is  needed  to  adapt  the  commands  and  data  from  the 
core  system  to  the  participant  and  to  perform  distributed  processing.  A 
transceiver  is  required  to  communicate  with  the  core  instrumentation  (and 
other  participants)  over  a  fully  secure  datalink.  A  small  portable 
display/debriefing  station  is  envisioned  for  individual  ship  participants. 

*  The  core  instrumentation  functions,  which  ships  and  fixed  ranges  can  be 
configured  to  perform,  include  processing  and  display  for  setting  up, 
controlling,  and  debriefing  the  entire  training  exercise,  in  addition,  the 
core  instrumentation  is  expected  to  manage  a  satellite  communications 
terminal  to  connect  the  deployed  TCTS  system  with  shore  facilities. 
However,  the  core  instrumentation  and  participating  aircraft/ ships  will  be 
capable  of  completely  autonomous  operations  when  not  in  communication  with 

facilities.  All  datalinks  will  be  fully  encrypted  and  secure. 

*  Interfaces  with  fixed  range  facilities  (e.g.,  Fallon  TACTS,  SCORE,  AFWTF, 
PMRF)  are  included  in  the  design  to  allow  data  to  be  exchanged  between  them 
and  the  TCTS  core  instrumentation  via  communications  satellite  for  display 
on  any  system.  TCTS  data  could  be  provided  to  the  fixed  ranges  and  data 
from  the  fixed  ranges  could  be  displayed  at  sea. 

*  Interfaces  with  shore-based  training  simulator  facilities  are  included 
as  a  growth  capability  to  support  their  future  integration  into 
TCTS-supported  exercises  and  participation  in  subsequent  debriefing. 

*  Remote  debriefing  facilities  are  included  in  the  design  to  allow  exercise 
data  to  be  displayed  at  any  desired  remote  location  and  for  more  convenient 
debriefing  of  distant  exercise  participants  as  required. 

*  Interfaces  with  a  shore  support  facility  are  also  included  in  the  design. 
The  shore  support  facility (s)  would  contain  processing  and  display  systems 
necessary  to  support  both  exercise  coordination  (for  large,  multi force 
exercises)  and  exercise  analysis  (for  Fleet  readiness  and  Battle  Group 
effectiveness  evaluation) . 
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TCTS  TOP-LEVEL  NOTIONAL  DESIGN 
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EXAMPLE  TCT8  BATTLE  GROUP  TRAINING  SCENARIO 


To  illustrate  the  concept  of  operations  for  employment  of  the  TCTS  to 
support  enhanced  combat  training  at  sea,  an  example  Battle  Group  training 
scenario  is  presented.  The  principal  functions  to  be  performed  by  the  major 
elements  of  the  TCTS  are  described  for  the  pre-mission  system 
initialization,  mission  operations,  and  post-mission  debrief  phases  of  the 
training.  The  example  training  exercise  involves  simultaneous  ASUW  and  ASW 
operations  conducted  by  a  CV  Battle  Group.  The  TCTS  core  instrumentation 
is  located  aboard  the  CV.  Alteratively,  the  core  instrumentation  could  be 
located  aboard  another  flag-configured  ship.  Each  participating  ship  and 
aircraft  is  equipped  with  an  onboard  TCTS  instrumentation  unit. 

Initialization 

Prior  to  commencement  of  the  exercise,  the  training  officer  interacts 
with  the  TCTS  core  instrumentation  to  set  it  up  for  the  planned  exercise 
operations.  Appropriate  ASUW  and  ASW  missions/scenarios  are  called  up  from 
the  stored  TCTS  database,  and  they  are  modified  combined,  and  tailored  to 
incorporate  the  specific  target  platform  configurations,  threat  system 
characteristics,  and  weapons  loadings  that  must  be  simulated  by  the  TCTS  to 
meet  the  training  objectives  of  this  exercise.  As  necessary  r  new 
mission/scenario  parameters  are  generated  and  entered  into  the  database. 

As  exercise  operations  begin,  the  TCTS  core  instrumentation 
automatically  establishes  secure  two-way  data  communications  with  each 
participant's  onboard  TCTS  instrumentation  unit  via  the  instrumentation 
datalink.  Once  TCTS  datalink  connectivity  is  established  with  each 
participant,  the  core  instrumentation  uploads  that  participant's 
mission/scenario  parameter  database  (targets,  threats,  weapons,  etc.)  to 
the  participant's  onboard  instrumentation  unit.  Since  the  TCTS  datalink 
is  fully  encrypted,  the  individual  participant  databases  that  are  uploaded 
over  the  air  can  include  classified  information. 

At  COMEX,  the  exercise  proceeds  without  interference  from  the  TCTS 
instrximentation,  which  performs  its  functions  and  manages  its  interfaces 
with  participants  automatically.  The  crews  aboard  exercise  participants 
perform  their  normal  tactical  functions;  no  additional  interactions  with 
TCTS  instrumentation  are  required. 
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EXAMPLE  TCTS  BATTLE  GROUP  TRAINING  SCENARIO 

INITIALIZATION 


•  BEFORE  EXERCISE  BEGINS,  TRAINING 
OFFICER  INTERACTS  WITH  TCTS 
CORE  INSTRUMENTATION: 

-  IDENTIFIES  DESIRED  MISSION/SCENARIO 

-  MODIFIES/TAILORS  DATABASE 
PARAMETERS  TO  SUIT  SPECIFIC  TRAINING 
OBJECTIVES 


TCTS  CORE  INSTRUMENTATION: 

-  ESTABLISHES  SECURE  DATALINK 
CONNECTIVITY  WITH  EACH  EXERCISE 
PARTICIPANT 

-  UPLOADS  MISSION/SCENARIO  PARAMETER 
DATABASE  (TARGETS.  WEAPONS.  THREATS 
etc)  TO  EACH  PARTICIPANT'S  ONBOARD 
TCTS  INSTRUMENTATION  UNIT 

EXERCISE  PROCEEDS  WITHOUT  INTERFERENCE  FROM 


surface  and  Airborne  Exercise  Participant  Operations 


As  the  Battle  Group  ships  and  aircraft  pursue  their  assigned  tactical 
missions  (e.g.,  the  ASUW  operation  is  illustrated),  each  participation 
onboard  TCTS  instrumentation  unit  continuously  acquires  position  data  either 
from  the  platform's  organic  GPS  receiver  (if  one  is  installed)  or  from  a  GPS 
receiver  incorporated  in  the  instrumentation  unit.  TCTS  instrumentation 
units  also  perform  a  similar  function  aboard  any  participating  air  or 
surface  platforms  that  are  employed  in  the  exercise  to  act  as  ORANGE  Force 
threat  elements.  When  a  simulated  threat  ship  or  aircraft  enters  the 
coverage  of  a  BLUE  participant,  that  participant's  TCTS  instrumentation  unit 
stimulates  its  tactical  sensors  and  combat  systems  with  targets,  threats, 
and  EW  in  accordance  with  the  training  scenario  database  stored  in  the 
instrumentation  unit.  When  a  "real"  threat  platform  (i.e.,  an  aircraft  or 
ship  that  is  emulating  a  specified  adversary  platform  type)  enters  the 
coverage  of  a  BLUE  participant,  that  participant's  TCTS  instrumentation  unit 
stimulates  its  tactical  systems  that  are  not  realistically  simulated  by  the 
adversary  platform  and  its  onboard  equipment.  The  TCTS  instrumentation  unit 
also  monitors  GPS-based  position  reports  transmitted  over  the  TCTS 
interparticipant  datalink  by  other  nearby  exercise  participants  and 
correlates  targets  identified  by  ownship  tactical  sensors  and  weapons 
systems  with  TCTS  reference  tracks  (both  GPS-based  real  tracks  and 
scenario-driven  simulated  tracks) .  The  instrumentation  unit  runs  real-time 
simulations  to  evaluate  the  outcome  of  any  weapon  employment  events 
initiated  by  the  BLUE  ship  or  aircraft  crew  and  provides  feedback  assessment 
(e.g.,  kill/no-kill)  to  the  crew.  It  also  assesses  the  results  of  any 
countermeasures  employed  by  the  crew  against  simulated  threats  and  modifies 
the  stimulation  characteristics  accordingly. 

Using  multiple  datalink  relays  through  TCTS-equipped  participants 
within  mutual  RF  line-of-sight,  the  TCTS  core  instrumentation  monitors 
participants'  positions  and  sensor/weapon  system  activity  (including 
stimulation,  weapons  simulation,  and  evaluation  of  results)  via  downlink 
exercise  data  messages.  The  core  instinimentation  updates  the  BLUE 
participants'  onboard  scenario  databases  (e.g.,  deletion  of  threat  elements 
evaluated  as  destroyed  by  other  participants)  via  uplink  messages  to 
accomplish  dynamic  real-time  coordination  of  the  ongoing 
simulated/stimulated  mission  scenario  among  all  the  exercise  participants. 

The  progress  of  the  mission  can  be  observed  in  real  rime  it  the  TCTS 
core  instrumentation  monitoring/control  station.  Throughout  the  mission, 
the  core  instrumentation  stores  exercise  data  for  use  in  post-exercise 
replay  and  debrief. 
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EXAMPLE  TCTS  BATTLE  GROUP  TRAINING  SCENARIO 

SURFACE  &  AIRBORNE  EXERCISE  PARTICIPANT  OPERATIONS 


•  EACH  PARTICIPANT'S  ONBOARD  TCTS  INSTRUMENTATION  UNIT : 

-  DfTffiM/NfS  PARTICIPANT'S  CPS  BASCD  POSITION 

-  monitors  position  reports  from  other  nearby  participants 

-  stimulates  participants  sensors  with  targets,  threats. 

&  EW  PER  TRAINING  scenario 

-  correlates  OWNSHIP  target  IDi  WITH  REFERENCE  TRACKS 

(RIAL  S  SIMULAICD) 

-  EVALUATES  ANY  WEAPON  EMPLOYMENT  EVENTS  &  PROVIDES 
FEEDBACK  assessment  TO  PARTICIPANT 


GPS  SATELLITE 
CONSTELLATION 


•  USING  DATALINK  RELAYS  THROUGH  EXERCISE  PARTICIPANTS,  TCTS 
CORE  INSTRUMENTATION: 

-  MONITORS  PARTICIPANTS'  POSITION  &  SENSOR/WEAPON  SYSTEM 
activity  (including  iiveARONUCOUNTCilMCASUliCS  evaluations) 

-  UPDATES/COORDINATES  COMMON  SIMULATED/STIMULATED 
SCENARIO  WITH  OTHER  PARTICIPANTS 


Detached  Cluster  Operations 


Some  exercise  missions  will  involve  BLUE  Force  participant  operations 
in  areas  where  insufficient  relay  paths  exist  to  maintain  continuous  TCTS 
datalink  connectivity  with  the  core  instrumentation  (e.g.,  outer-zone  ASW 
operations  are  illustrated).  However,  each  participant's  onboard  TCTS 
instrumentation  unit  performs  the  position  and  event  monitoring, 
stimulation,  weapon  simulation,  and  evaluation  functions  described 
previously.  Interparticipant  communications  via  the  TCTS  secure  datalink 
are  used  to  accomplish  dynamic  real-time  coordination  of  the 
Simula ted/ stimulated  mission  scenario  within  the  detached  cluster  of 
participants.  If  real  targets  or  signals  are  present,  they  will  also  be 
processed  by  participants'  tactical  sensors  and  combat  systems;  the  TCTS 
stimulation  techniques  do  not  preclude  detection  and  acquisition  of  real 
targets  by  the  sensors  and  combat  systems  being  stimulated. 

While  operating  beyond  datalink  connectivity  with  the  core 
instrumentation,  each  onboard  TCTS  instrumentation  unit  stores  significant 
mission  exercise  data  (time-tagged  positions,  events, 
stimulation/simulation/weapons  data,  and  evaluation  results)  for  later 
downlink  transmission  to  the  core  instrumentation  when  TCTS  datalink 
connectivity  is  reestablished. 
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EXAMPLE  TCTS  BATTLE  GROUP  TRAINING  SCENARIO 

DETACHED  CLUSTER  OPERATIONS 


•  WHEN  BEYOND  DATALINK  CONNECTIVITY  WITH 

TCTS  CORE  INSTRUMENTATION,  EACH  PARTICIPANTS 
ONBOARD  TCTS  INSTRUMENTATION  UNIT: 

-  PERFORMS  SAME  FUNCTION  AS  WHEN  CONNECTIvny 
EXISTS  (monitors,  stimulates,  a  evaiuates) 

-  COORDINATES  COMMON  SIMULATED/STIMULATED 
SCENARIO  WITH  OTHER  PARTICIPANTS  (updates  SCENARIO 

OaTABaSE-  c  g,  TO  MARK  A  THREAT  •KILLED') 

-  RECORDS  SIGNIFICANT  DA  TA  FOR  LA  TER  DOWNLINK 

TO  CORE  INSTRUMENTATION  fTIME/POSITION/EVENT/FIESULT) 


GPS  SATELLITE 
CONSTELLATION  cQr 


STIMULATION  METHOD  DOES  NOT  PRECLUDE  ONBOAR 
SENSORS'  ABILITY  TO  ACQUIRE  REAL  TARGETS 
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Data  Retrieval  from  Submarines 


When  submarines  participating  in  TCTS-supported  exercises  operate 
within  the  instrumentation  coverage  of  fixed  underwater  ranges,  *  real-time 
three-dimensional  track  data  are  available  for  submarines,  surface  ships, 
and  unden/ater  weapons  equipped  with  acoustic  pingers.  These  tracks  are 
relayed  in  real  time  via  satellite  from  the  range  operations  center  to  the 
TCTS  core,  instrumentation  where  they  are  integrated  with  the  track  and 
event  data  collected  from  exercise  participants  via  the  TCTS  datalink. 

When  submarines  operate  off-range  in  the  open  ocean,  time-tagged 
underwater  track  and  event  data  are,  collected  by  the  TCTS  core 
instrumentation  on  a  post-event  basis  via  one  of  two  methods: 

(1)  The  submarine  comes  to  communications  depth  and  transmits  data 
(e.g.,  via  satellite  relay  channel) 

(2)  The  submarine  loads  data  into  a  SLOT-buoy-type  device  (e.g., 
AN/BRT— 6  SATCOM  buoy)  and  releases  the  device  from  operating  depth. 
The  device  ascends  to  the  surface  and  transmits  data  via  the 
Submarine  Satellite  Information  Exchange  Subsystem  (SSIXS)  channel. 

In  either  case,  the  satellite  relay  transmissions  may  be  received 
directly  by  the  TCTS  core  instrumentation  or  they  may  be  relayed  to  the 
TCTS  core  instrumentation  by  the  cognizant  Submarine  Operating  Authority 
(SUBOPAUTH) . 


*  Or  portable  underwater  tracking  ranges  such  as  are  under  development  by 
the  Naval  Undersea  Systems  Center  (NUSC) . 
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EXAMPLE  TCTS  BATTLE  GROUP  TRAINING  SCENARIO 

DATA  RETRIEVAL  FROM  SUBMARINES 


-  RELEASE  OF  SLOT- BUOY-TYPE  DEVICE  FROM 
SUBMARINE  OPERATING  DEPTH 
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Debrief 


Upon  completion  of  the  exercise,  the  training  officer  interacts  with 
the  core  instrumentation  to  initiate  the  final  correlation  and 
integration  of  all  exercise  data  collected  by  the  TCTS.  A  consolidated 
mission  history  database  is  generated  by  the  core  instrumentation  and 
transmitted  via  communications  relay  satellite  to  all  of  the  locations 
afloat  and  ashore  where  participating  personnel  are  located.  This 
mission  history  database  is  then  used  at  each  location  to  provide 
exercise  replay  and  debrief,  either  on  portable  TCTS  display/debriefing 
stations  or  on  available  installed  display  systems  via  TCTS-provided 
interfaces.  The  exercise  replay/debrief  presentation  can  also  be  viewed 
at  the  TCTS  core  instnmentation  monitoring/control  station.  Potential 
ashore  recipients  of  the  mission  history  database  for  this  example  Battle 
Group  exercise  include  the  home  naval  air  station  of  participant  VP 
aircraft,  the  range  operations  center  at  the  underwater  range,  and  the 
Submarine  Operating  Authority  (SUBOPAUTH) ,  which  can  relay  debrief  data 
to  participant  submarines  via  appropriate  submarine  force  communications 
channels  if  direct  reception  of  satellite-relayed  TCTS  debrief  data  by 
the  submarines  is  precluded  by  their  operating  schedules. 
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EXAMPLE  TCTS  BATTLE  GROUP  TRAINING  SCENARIO 

DEBRIEF 


•  CORE  INSTRUMENTATION  PROVIDES  CONSOLIDATED 
MISSION  HISTORY  LOG  TO  PARTICIPANTS 
&  SHORE  FACILITY(s)  FOR  REPLAY/DEBRIEF 
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TOTS  CONCEPT  OF  OPERATIONS— -SUMMARY 


This  short  executive  overview  can  be  summarized  by  these  three  major 
points : 

First — Tactical  training  capabilities  for  deployed  forces  must  be 
improved,  emphasizing  all  levels  of  training  from  individual  platforms 
through  battle  group/force  and  collecting  the  data  needed  to  support 
assessment,  tactics  development,  and  OT&E. 

Second— -The  requirements  for  a  TCTS  emphasize  greater  realism  in  the 
combat  training  environment,  accurate  assessment  of  multiparticipant 
tactical  training  engagements,  and  comprehensive  event  monitoring  along 
with  timely  interactive  feedback  for  the  individual  units  undergoing  the 
training. 

Third — Available  technology  can  be  used  to  meet  the  TCTS 
requirements  and  provide  enhanced  training  capability  to  the  Fleet. 
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TCTS  CONCEPT  OF  OPERATIONS 

SUMMARY 


NAVY  NEEDS  IMPROVED  TACTICAL  TRAINING  CAPABILITY  FOR 
DEPLOYED  FORCES 

-  INDIVIDUAL  PLATFORM  THROUGH  BATTLE  GROUP/FORCE  LEVEL 

-  SUPPORT  READINESS  ASSESSMENT,  TACTICS  DEVELOPMENT, 

&OT&E 

TCTS  MUST  PROVIDE: 

-  GREATER  REALISM  IN  THE  COMBAT  TRAINING  ENVIRONMENT 

-  ACCURATE  ASSESSMENT  OF  MULTIPARTICIPANT  TACTICAL 
TRAINING  ENGAGEMENTS 

-  COMPREHENSIVE  EVENT  MONITORING  &  TIMELY  INTERACTIVE 
FEEDBACK 


AVAILABLE  TECHNOLOGY  CAN  MEET  THE  REQUIREMENTS 


"Embedded  Training  Capabilities  for  the  LAMPS  MK  III  System" 

By 

Robert  S.  Romalewski,  NOARL 
James  Hammond,  NOARL 
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"  Embedded  Training  Capabilities  for  the  LAMPS  MK  III  System" 
Robert  S.  Romalewski  and  James  H.  Hammond 

Abstract: 

An  SH-60  helicopter  incorporation  the  Light  Airborn  Multipurpose 
System  (LAMPS)  MK  III  was  tested  using  the  AN/SRQ-4  UHF  secure  data 
link  to  an  AN/SQQ-28  Sonobuoy  Processor  at  the  Naval  Air  Test 
Center  (NATC)  Patuxent  River,  MD.  During  the  time  the  helicopter 
was  on  the  simulated  mission,  it  "dropped"  sonobuoys  and  received 
normal  mission  feedback.  This  was  done  during  testing  of  a 
potential  wide-area  network,  with  the  acoustic  stimulation  for  the 
LAMPS  MK  III  accomplished  by  passing  control  and  acoustic 
information  from  an  AN/SQQ-89  On-Board  Trainer  (OBT)  at  Patuxent 
River  via  an  on-site  data  link  network.  A  master  control  unit  was 
used  to  drive  NATC's  OBT  from  Fleet  ASW  Training  Center,  Norfolk, 
over  normal  telephone  lines.  Multiple  OBT  connections  from  the 
master  control  unit  are  possible  and  will  further  team  training  by 
combining  expertise  located  at  another  training  center  with  all 
other  connected  assets.  Other  additions  to  the  OBT  interface  will 
provide  additional  aircrew  radar  training  via  the  AN/APS-124  Remote 
Radar  Operator  (REMRO)  system  and  EW  software  upgrades.  The  OBT  is 
used  at  NATC  in  evaluation  of  helicopter  ASW  subsystems  as  well  as 
for  training. 

Overview: 

The  LAMPS  MK  III  aircrew  currently  benefits  from  an  extensive 
embedded  training  capability  using  the  AN/SQQ-89 (V) -T()  On-Board 
Trainer.  This  trainer  provides  both  air  and  ship  crew  ASW  training 
utilizing  an  actual  SH-60  helicopter  in  the  air  or  on  deck  as  well 
as  a  simulated  helicopter  mode  for  training  ship's  crew.  The 
aircrew  can  perform  all  functions  necessary  to  prosecute  a  complete 
ASW  problem  including  placing  sonobuoys,  operating  with  the  Air 
Tactical  Operator  (ATACO) ,  and  using  their  Magnetic  Anomaly 
Detector  (MAD) .  Figure  1  shows  the  environmental  and  tactical 
system  affected  by  the  OBT  and  LAMPS  MK  III.  The  LAMPS  MK  III 

system  is  tied  to  the  AN/SQQ-28  sonobuoy  signal  processing  system, 
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via  the  data  link.  A  complete  LAMPS  MK  III  sonobuoy  inventory  is 
available. 

Recent  R&D  sponsored  by  the  Surface  Ship  ASW  Combat  Systems  Program 
Office  will  update  the  Electronic  Warfare  Simulation  (EWSIM) 
program  and  add  capability  to  train  the  AN/APS-124  Radar  Operator 
and  Remote  Radar  Operator  on  board  ship.  An  upgrade  from  a  single 
ship  and  single  LAMPS  MK  III  helicopter  trainer  to  a  Multi-Ship  - 
Multi-Environment  Networked  Training  and  Operational  Readiness 
system  (MS-MENTOR)  wherein  multiple  ships  and  aircraft  are  trained 
in  a  battle  group  configuration  is  currently  underway. 

Technical  Par2uneters : 

The  OBT  currently  simulates  EW  targets  for  both  the  AN/SLQ-32 
shipboard  EW  Receiver  and  the  LAMPS  MK  III  AN/ALQ-142  EW  Receiver. 
The  OBT  also  handles  acoustics  such  as  the  AN/SQS-53  hull-mounted 
sonar,  the  AN/SQR-19  towed  array,  and  AN/SQQ-28  LAMPS  MK  III 
sonobuoy  processor,  which  are  stimulated  through  an  inverse 
beamformer.  The  main  thrust  of  the  new  development  are  to  extend 
the  current  operational  system  via  MS-MENTOR,  partially  test 
MS -MENTOR  as  the  scenario  controller,  and  incorporate  added 
features  for  the  SH-60  helicopter  crew.  In  the  OBT,  EW  and  ASW 
targets  are  generated  in  a  gaming  area  of  2048  x  2048  nautical 
miles,  to  an  altitude  of  100,000  feet  and  to  a  depth  of  5,000  feet. 
These  targets  are  then  position-kept  in  the  OBT,  and  echo/ emissions 
become  visible  to  the  sensor  operators  when  the  signal-to-noise 
ratio  of  these  targets  is  such  that  it  exceeds  the  detection 
threshold  of  the  sensor  systems.  The  ESM  Contact  Generation 
function  of  the  OBT  has  the  ESM  processor  manage  and  control  the 
injection  of  ESM  emitter  parameters  into  the  shipboard  (AN/SLQ-32) 
and  LAMPS  MK  III  (AN/ALQ-142)  sensors.  Radars,  communications 
emissions,  data  links,  and  weapon  seekers  are  all  provided  in  a 
geo-tailored  ESM  library  of  emitters.  Helicopter  position  data  is 
utilized  for  platform  visibility  and  angle  of  arrival  as  part  of 
the  position  information.  The  ESM  processor  continuously  computes 
(at  a  4  Hz  rate)  for  update  transmission  to  the  sensors  these  last 


two,  plus  signal  propagation  loss,  ducting  values,  and  parameter 
changes . 

The  LAMPS  Tactical  Data  Simulation  (TDS)  processor  in  the  OBT 
communicates,  via  a  standard  NTDS  Type  A  (Slow)  interface,  with  the 
AN/SQQ-28  Shipboard  Processor  Operation  Program  for  the  purpose  of 
exchanging  sonobuoy  and  LAMPS  MK  III  status  information.  TDS  also 
PJ^ovides  for  Magnetic  Anomaly  Detector  Simulation,  ordnance 
management,  sonobuoy  management,  and  ESM  message  routing  to  the 
helicopter.  Were  the  helicopter  not  available,  the  TDS  would 
role-play  in  its  stead.  In  normal  operation,  the  LAMPS  MK  III  ESM 
air  subsystem  is  controlled  from  the  AN/SLQ-32  by  the  ESM  operator, 
and  all  ESM  data  is  downl:;.nked  for  analysis  and  display.  The  data 
link  also  provides  transmission  of  tactical  instructions,  weapon 
delivery  information,  and  other  processed  data  (such  as  digitized 
voice  communications)  back  to  the  helicopter.  Feedback  to  the 
LAMPS  helicopter  allows  for  a  variety  of  scenarios  and  crew 
training. 

LAMPS  MK  III  AM/APS-124  and  Remote  Radar  Operator  (REMRO)  Training: 
The  AN/APS-124  Radar  Operator  aboard  the  helicopter  as  well  as  the 
Remote  Radar  Operator  aboard  ship  manning  an  OJ-194  console  do  not 
receive  formal  training.  The  LAMPS  MK  III  Fleet  Project  Team 
identified  this  training  deficiency  four  years  ago,  and  it  has 
since  been  validated  by  OPNAV  letter  with  an  OBT  requirements 
change  to  add  the  requirement  for  OBT  to  provide  embedded  training 
for  these  functions. 

There  is  an  ongoing  proof-of -concept  to  determine  the  validity  of 
the  approach  to  add  this  training  feature.  The  proposed  approach 
to  add  an  APS— 124/REMRO  simulation  unit  to  be  temporarily 
installed  aboard  the  LAMPS  MK  III  helicopter  which  would  feed  a 
simulated  radar  picture  into  the  output  of  the  APS-124.  This  block 
diagram  is  shown  in  figure  2.  A  trade-off  study  done  by  Raytheon 
of  whether  a  radar  simulator  located  on  the  ship  with  uplink  radar 
video  or  a  portable  radar  simulator  that  could  be  mounted  in  either 


the  helicopter  or  ship  was  done  in  FY  91.  The  study  found  the 
portable  system  the  most  cost  effective  solution.  Figure  3  shows 
the  location  of  the  temporary  unit  on  the  helicopter.  Initially, 
coordinated  ASW/EW/radar  target  features  would  be  provided,  but  not 
radar  landmass.  The  proof-of-concept  will  be  completed  at  the  Ship 
Ground  Station  at  NATO  Patuxent  River  by  Feb  92.  Funding  to 
implement  this  feature  will  be  dependent  on  the  budgetary  process 
for  FY92  and  outyears. 

The  need  for  coordinated  team  radar  training,  coming  from  air  team 
feedback,  requires  an  upgrade  to  the  OBT  to  provide  a  LAMPS  MK  III 
radar  training  capability  and  integrated  acoustic  and  ESM  training. 
Part  of  the  proposed  functionality  is  the  operation  of  the 
AN/APS-124  LAMPS  MK  III  airborne  search  radar  to  detect  targets 
beyond  own  ship's  capability,  including  low  flying  aircraft  and 
anti-ship  missiles.  Other  aspects  include  intercommunications  with 
other  MK  III  tactical  team  members  and  units;  operator  airborne  IFF 
interrogation  to  challenge  unidentified  contacts;  correlating  EW 
information  with  MK  III  tracks;  entering  target  data  generated  for 
display  and  analysis,  and  updating  all  displays. 

Electronic  Warfare  Simulation  (EWSIH)  Upgrades: 

There  is  a  special  program  that  is  part  of  the  LAMP  MK  III  Airborne 
Operational  Program  (AOP)  called  EWSIM  that  is  loaded  for  the  EW 
training  mode  in  the  helicopter.  This  program  is  currently  being 
upgraded  by  the  Naval  Air  Development  Center  (NADC)  to  add 
helicopter  threat  warning  capability  and  to  make  other  software 
improvements.  This  program  upgrade  will  be  tested  in  Feb  92,  and 
will  probably  be  made  part  of  AOP  fleet  release  20  or  21. 

OBT  ASW  Training  Capabilities  Applied  to  Non-LAMPS  Platforms: 
Directional  Frequency  Analysis  and  Recording  (DIFAR)  scenarios 
require  no  interface  between  the  trainer  and  the  receiving 
platform.  In  the  currently  prototyped  "School  House"  mode,  OBT 
generated  DIFAR  scenarios  can  be  used  by  non-LAMPS  ASW  platforms. 
Active  scenarios,  however,  requir^^that  the  OBT  receive  returned 


"trigger"  signals  to  denote  initiation  of  active  sonar  pings  by  the 
airborne  platform.  NATO  engineers  have  conceptualized  a 
modification  which  would  provide  a  means  of  receiving  Directional 
Command  Activated  Sonobuoy  (DICAS)  ping  trigger  signals  from 
non-LAMPS  aircraft.  This  modification  would  improve  the  OBT's 
utility  by  providing  both  passive  and  active  acoustic  training 
support  for  the  P-3,  S-3  fixed  wing,  and  the  SH-2F,  SH-2G,  SH-3H, 
SH-60F  or  non-LAMPS  MK  III  helicopter  communities.  If  fleet 
interest  is  found  to  exist,  funding  will  be  sought  for  development 
of  a  prototype  ping  trigger  receiver/decoder.  This  modification 
would  broaden  the  applicability  of  the  AN/SQQ-89  trainer,  and  would 
further  enhance  its  usefulness  for  support  of  integrated  battle 
force  ASW  training. 

Non-Training  Applications  of  the  AN/SQQ-89/OBT: 

Engineers  at  the  Naval  Air  Test  Center,  Patuxent  River,  MD  are 
making  use  of  the  AN/SQQ-89  On  Board  Trainer  in  areas  outside  the 
training  arena.  As  an  example,  OBT  Directional  Frequency  Analysis 
and  Recording  acoustic  scenarios  were  transferred  to  tape.  These 
scenarios  were  then  used  to  support  Navy  evaluation  of  the  SH-2G 
helicopter  integrated  ASW  mission  system.  The  OBT  was  also  used  to 
provide  ASW  scenarios  for  the  SH-60B  helicopter  system  software 
developmental  test  and  evaluation.  Additionally,  the  OBT  provides 
acoustic  stimulation  during  SH-2F,  SH-2G  and  SH-60F  helicopter  ASW 
test  and  evaluation  as  well  as  the  SH-60B. 

Planning  is  underway  to  develop  procedures  that  will  permit 
deployed  LAMPS  MK  III  system  maintenance  personnel  to  verify 
integrated  ship/air  acoustic  processing  performance  using  the  OBT 
as  an  acoustic  signal  source.  This  application  will  provide  both 
ships  company  and  embarked  aircraft  maintenance  personnel  with  a 
means  of  evaluating  sonobuoy  receiver,  acoustic  processor,  and 
display  performance  using  signal  levels  which  approach  the 
detection  threshold. 
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MS-MENTOR: 

The  Fleet  ASW  Training  Center  Atlantic  in  Norfolk,  Virginia,  is  the 
evaluation  testbed  for  the  Multi-Ship  -  Multi-Environment  Networked 
Training  and  Operational  Readiness  system:  a  Raytheon  IR/IRED 
initiative,  which  controls  multiple  AN/SQQ-89  On-Board  Trainers  for 
ASW  work.  This  control  is  set  up  via  a  network  construct  termed 
OBT-NET,  and  supports  the  Battle  Force  Tactical  Training  (BFTT) 
program.  The  Ship  Ground  Station  at  the  Naval  Air  Test  Center 
serves  as  a  link  to  the  OBT  and  LAMPS  MK  III.  Battle  Force 
Tactical  Training  units  in  the  Chesapeake  Bay  -  Norfolk  area  are 
another  potential  network  linkage.  A  mobile  trailer  configuration 
for  the  host  node  which  allows  the  MS-MENTOR  configuration  to  be 
transported  to  other  ports  of  opportunity  has  been  developed  by  the 
Naval  Oceanographic  and  Atmospheric  Research  Laboratory  (NOARL) . 

The  main  thrust  of  our  new  development  was  to  extend  the  current 
operational  system  by  adding  multi-ship  networking,  and  partially 
test  MS-MENTOR  as  the  scenario  controller.  Evaluation  of  the  last 
component,  MS-MENTOR,  is  ongoing  under  the  auspices  of  the  Naval 
Underwater  Systems  Center  (NUSC) .  Multiple  OBT  connections  from 
this  master,  control  unit  are  possible  and  will  further  team 
training  and  asset  coordination.  Even  if  only  utilized  on  a 
limited  basis,  it  can  provide  training  for  crew  and  linked  ASW  team 
members,  combined  with  expertise  located  at  another  training 
center.  In  this  specific  case,  a  LAMPS  MK  III  helicopter  at  NATC 
was  connected  through  the  AN/SRQ-4  UHF  data  link  to  the  OBT  at 
NATC,  and  the  MS-MENTOR  system  in  Norfolk.  Monitoring  of  the 
exercise,  and  by  extension  any  multiple  air-sea  platform  grouping, 
was  done  at  the  scenario's  central  control  in  Norfolk,  as  well  as 
at  NATC.  The  MS-MENTOR  system  can  currently  provide  external 
control  of  up  to  eight  OBT's,  and  can  itself  be  slaved  to  an 
external  controller  through  BFTT.  Figure  4  shows  a  schematic  of 
the  MS-MENTOR  test  configuration  to  the  other  players. 

The  external  control  for  the  scenario  could  be  another  higher  level 
training  system,  such  as  the  AEGIS  Combat  Training  System  (ACTS) , 


which  in  turn  would  incorporate  Anti-Surface  Warfare  (ASUW)  to  the 
Anti-Air  Warfare  (AAW)  and  Anti-Submarine  Warfare  (ASW) .  The 
airborne  ESM  system  of  the  LAMPS  MK  III  can  be  viewed  as  an 
extension  of  the  shipboard  EW  system.  The  advantage  of  the 
MS-MENTOR  unit  lies  in  its  capability  to  tie  several  OBT's  and 
their  associated  network  components  together.  A  NTDS  Link  11 
implementation  using  ACTS  to  other  OBT's  would  be  limited  to  only 
ships  having  the  AEGIS  Combat  Weapons  System,  whereas  the  MS-MENTOR 
approach  would  allow  all  AN/SQQ-89  ships  having  an  OBT  to 
participate.  The  OBT  will  contain  an  interface  to  ACTS  with  the 
release  of  AEGIS  Baseline  4,  and  has  an  external  control  port  for 
stimulation  control  from  sources  such  as  the  20B5  FFG7  Pierside 
Team  Trainer.  The  existence  of  an  external  control  port  and  an 
associated  software  design  that  allows  for  OBT  control  from  an 
outside  source  will  allow  BFTT  to  interface  with  the  OBT  with  no 
modifications  required  to  the  existing  OBT. 

In  order  to  pass  information  within  the  MS-MENTOR  structure  and 
among  the  dissimilar  components,  network  interoperability  is 
required.  An  open  system  computer  network  architecture,  which  is 
supported  by  the  services  and  industry,  that  uses  standard 
protocols  is  nearing  its'  final  draft  form  in  the  Distributed 
Interactive  Simulation  (OIS)  protocols,  and  will  be  released  as  an 
IEEE  standard.  It  supports  the  simulation  devices  being  present  in 
one  location,  interconnected  by  a  Local  Area  Network  (LAN) ,  or 
widely  distributed  on  a  Wide  Area  Network  (WAN)  .  As  the  simulation 
network  is  expanded,  each  new  player  or  functional  unit  brings  with 
it  all  of  the  computational  resources  necessary  to  support  itself. 
This  fits  into  BFTT  and  the  MS-MENTOR  configuration. 

Summary: 

We  have  talked  about  the  embedded  training  capabilities  of  the 
LAMPS  MK  III  and  OBT  systems.  The  OBT  is  used  for  more  than  just 
training  at  NATO,  and  has  not  reached  the  end  of  its  usefulness 
yet.  Within  the  BFTT  arena,  initial  tests  of  the  MS-MENTOR  were 
successful,  and  it  was  shown  that  resources  throughout  a  wide  area 


can  operate  together  for  team  training.  Future  additions  to  Navy 
resources  will  be  the  DIS  tri-service  computer  interoperability 
protocols,  EW  upgrades,  and  REMRO  capabilities. 
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AN/APS-ia4 


Figure  2.  Radar  Simulation  Unit  Connection  Diagram  (Air  Team  Training  Configuration) 
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AIRCRAFT  LAYOUT 
Figure  3.  SH-60  Internal  View. 


Naval  Air  Test  Center,  Patuxent  River,  MD 
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FLEASWTRACENLANT 
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Figure  4.  OBT-NET  and  MS-MENTOR  Connectivity. 
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The  problems  of  developing  Computer-Based  Training  (CBT)  and  training  in 
general  get  more  difficult  as  we  push  on  to  the  21st  century.  The  tasks  that  we 
are  being  asked  to  develop  training  for  are  becoming  more  technical  and 
sophisticated  while  our  audience  and  our  developers  become  relatively  less 
capable.  In  an  Executive  Summary  published  in  1987  by  the  Hudson  Instimte 
entided  Workforce  2000,  Work  and  Workers  for  the  Twenty-First  Century  they  list  six 
challenges  facing  American  jobs  and  workers  between  now  and  the  year  2000. 
These  six  issues  are: 

1 .  Stimulating  Balanced  World  Growth 

2.  Accelerating  Productivity  Increases  in  Service  Industries 

3 .  Maintaining  the  Dynamism  of  an  Aging  Workforce 

4.  Reconciling  the  Conflicting  Needs  of  Women,  Work  and  Families 

5.  Integrating  Black  and  Hispanic  Workers  Fully  Into  the  Economy 

6.  Improving  the  Education  and  Skills  of  All  Workers 

All  of  these  challenges  effect  us  in  one  way  or  another  but  the  one  that  we  can 
have  a  more  direct  impact  on  is  the  last  one,  Improving  the  Education  and  Skills 
of  All  Workers.  Clearly  this  challenge  needs  to  be  met  while  tomorrow’s 
workers  are  in  today’s  schools.  However,  this  may  or  may  not  happen  and  we 
will  be  forced  to  meet  this  challenge  in  the  workplace,  regardless.  As  the  need 
for  higher  skilled  employees  increases  we  will  have  to  be  more  creative  in  the 
development  of  our  training  as  well  as  in  the  tools  that  we  use  to  develop  that 
training.  As  the  gap  between  the  existing  skills  and  abilities  of  employees  versus  , 
the  requirements  of  advanced  technology  widens  our  challenge  will  be  ever 
greater.  As  systems  get  more  technologically  sophisticated  we  will  have  to  train 
our  employees  to  utilize  and  maintain  those  .systems.  The  skills  of  these 
employees  will  have  to  be  learned  on  the  job  in  many  cases,  through  the  training 
that  we  provide. 

As  trainers,  we  will  have  to  become  more  sophisticated  in  the  development  of 
our  training  and  the  use  of  the  training  tools  available  to  us.  Training  technology 
continues  to  advance.  Computer-based  training  has  been  around  for  many  years 
but  is  only  now  starting  to  reach  a  wider  audience  and  acceptance.  We  must 
continue  to  break  ground  in  our  use  of  the  newer  training  technologies. 
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This  means  that  we  have  to  use  more  advanced  tools  in  the  development  of  our 
courseware.  As  CBT  is  accepted  and  utilized  the  call  for  more  (-BT  will 
increase.  In  developing  all  this  interactive,  mediated  instruction  we  are  faced 
with  many  problems.  I  have  noted  one  problem,  the  less  skilled  worker  and 
developer.  With  more  courseware  being  developed  comes  the  problems  of  not 
only  developing  large  amounts  of  courseware  but  also  maintaining  that 
courseware.  How  do  you  manage  such  tasks?  How  can  you  streamline  the 
standardization  of  a  courseware  project?  How  can  you  take  advantage  of  the 
technology  to  better  organize  the  project?  How  can  you  make  your  development 
staff  more  productive?  Are  there  tools  available  to  help  with  these  problems?  1 
believe  that  there  are. 

One  of  the  most  significant  advances  in  training  development  technology  in 
recent  years  is  called  “lesson  generation.”  The  philosophy  behind  lesson 
generation  is  to  take  as  many  decisions  as  possible  concerning  a  course  and 
capture  those  in  project-wide  software  models.  Some  of  those  decisions  include 
use  of  graphics  and  color,  learner  control,  instructional  design,  and  audio  and 
video.  Capturing  more  and  more  of  these  definitions  on  a  project-wide  basis  you 
cut  down  on  the  number  of  subjective  decisions  on  all  these  dimensions  to  be 
made  by  individual  developers.  This  has  the  effect  of  increasing  consistency, 
policy,  instructional  rigor  and  overall  quality  while  at  the  same  time  it  greatly 
improves  productivity  and  maintainability. 

The  interactivity  and  instructional  sophistication  of  a  course  can  be  approximated 
by  the  level  of  branching  complexity.  Learner  control  is  very  important  to  the 
student’s  ability  to  explore  and  make  the  most  of  the  training  e.xperience.  The 
greater  the  learner  control  the  more  accommodating  this  is  to  varying  learner 
styles  and  abilities.  Having  greater  learner  control  allows  one  course  to  cover  a 
wider  range  of  needs.  Many  lessons  have  been  considered  complex  if  they 
allowed  4-6  branches  per  frame.  In  large  measure,  learner  control  has  been 
dictated  by  the  flexibility  of  the  tool  being  used  to  develop  the  courseware. 
Traditional  authoring  systems  usually  require  that  all  branches  be  made  by  hand 
and  then  debugged  and  tested  in  the  same  manner.  This  makes  a  rich  learner 
control  environment  very  expensive.  Using  lesson  generation  you  can  define  all 
of  the  learner  control  globally  for  the  project.  The  branching  can  then  be 
debugged  and  tested  one  time.  In  a  course  developed  using  lesson  generation  for 
SAS  for  Boeing  767  aircraft,  the  level  of  branching  complexity  from  any  given 
frame  varied  from  20-28  choices.  This  allowed  the  students  a  great  deal  of 
freedom  and  made  their  training  more  effective.  Overall,  lesson  generation  gives 
you  greater  productivity  and  consistency  and  you  can  aspire  to  more  learner 
control  than  traditional  methods  without  increasing  the  development  cost. 
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For  example,  when  the  project  staff  decides  on  the  location  of  screen  icons  tor 
learner  control  you  can  capture  that  one  time  in  the  “course  level”  software  so  no 
developer  will  ever  have  to  specify  these  for  any  screen  of  any  lesson.  These 
will  then  be  supplied  automatically  in  all  lessons.  Therefore,  you  cut  down  on 
the  amount  of  time  for  debugging  and  testing  of  courseware.  In  the  future  if  you 
want  to  change  the  look  or  location  of  the  icons  (or,  perhaps,  more  fundamentally, 
the  “behavior”  of  the  course)  it  is  a  simple  matter  to  change  it  in  one  or  two 
places  and  those  changes  can  be  reflected  in  a  matter  of  moments  throughout  the 
entire  course.  You  would  not  have  to  debug  or  test  the  course  to  make  those 
global  changes.  This  helps  cut  down  the  costs  of  CBT  development  significantly. 

This  also  helps  the  student  because  they  can  always  depend  on  certain  things 
being  in  the  same  place  and  behaving  the  same  way  every  time.  How  many  times 
have  you  worked  with  a  new  piece  of  courseware  and  wondered  how  to  exit  this 
program  or  make  something  happen  that  was  similar  in  another  program,  but  is 
slightly  different  in  this  one?  You  have  to  continually  relearn  and  assimilate 
those  differences.  By  making  global  decisions  you  cut  down  on  that  learning 
process  leaving  more  time  and  energy  available  for  the  real  training  task. 

Lesson  generation  also  allows  the  different  development  tasks  to  go  on 
simultaneously,  such  as  graphics  development,  logic  development,  and  video  and 
audio  development.  This  increases  the  overall  productivity.  Because  one 
product  is  used  for  the  entire  development  process,  communication  between  the 
designers,  developers  and  subject-matter  experts  (SME’s)  is  increased.  Figure  1 
shows  how  the  separation  of  content  and  instructional  strategy  with  lesson 
generation  may  be  used. 
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Separation  of  Content  and  Strategy 
with  Generation 
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As  computer  technology  continues  to  get  more  sophisticated  the  user-interface 
has  gotten  easier.  This  helps  with  the  problem  of  the  lower-skilled  developer. 
No  longer  do  we  require  a  computer  programmer  to  develop  our  CBT,  thanks  to 
authoring  systems,  and  even  they  have  become  more  user-friendly  through  the 
use  of  highly  graphical,  icon-oriented  systems.  We  can  now  import  highly 
realistic  scanned  images  and  use  a  combination  of  still  and  motion  video  to  help 
in  our  training  so  we  require  less  skill  in  graphics  development.  Through  the  use 
of  data  bases  and  electronic  communications  organizing  and  managing  a  large 
project  can  be  much  easier. 


One  of  the  most  difficult  tasks  at  the  beginning  of  any  large-scale  courseware 
development  project  comes  in  the  establishment  of  standards  for  instruction  , 
graphics,  use  of  media  such  as  audio/video  and  general  organization.  It  is 
absolutely  vital  to  a  students  success  that  these  problems  be  ironed  out. 
Otherwise,  the  smdent  is  constantly  battling  the  courseware  and  learning  in  spite 
of  it,  not  because  of  it. 
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Early  in  the  development  of  the  project  a  critical  issue  is  the  standardization  of 
instruction.  One  area  that  has  lagged  behind  in  the  development  of  authoring 
systems  and  tools  is  in  the  incorporation  of  quality  instructional  design.  The 
focus  has  been  on  the  computer  and  the  interface.  We  have  relied  on  in-house 
instructional  design  expertise,  if  we  had  it  at  all,  to  solve  the  instructional 
problems.  Having  the  appropriate  talent  on  hand  for  a  large-scale  project  can  be 
very  expensive,  by  the  time  you  have  all  the  project  development  staff,  subject- 
matter  experts,  etc.  Quite  often  the  instructional  development  is  left  to  the  SME 
who  may  not  be  qualified  to  make  instructional  decisions.  If  you  have 
instructional  designers  on  staff  they  are  usually  not  all  going  to  produce 
courseware  at  the  same  level  of  quality  or  ability.  Very  often  the  instructional 
design  in  your  course  becomes  a  product  of  the  lowest  common  denominator  or 
ability  of  the  least-skilled  designer.  To  ensure  instructional  rigor  requires  a  large 
staff  of  people  to  manage  and  review  all  of  the  work  done  by  these  instructional 
designers,  thereby  increasing  the  costs  while  not  guaranteeing  the  quality. 

A  system  that  could  take  advantage  of  available  in-house  instructional  expertise  as 
well  as  incorporate  the  knowledge  of  the  country’s  leading  experts  could  prove 
invaluable  to  a  project.  One  such  system  that  does  this  is  called  Sage.  Sage 
allows  you  to  capture  the  expertise  of  your  best  designers  concerning 
instructional  strategies,  learner  control  functions,  standard  screen  layouts  and 
control  and  mapping  icons.  This  expertise  can  then  be  incorporated  into 
instructional  templates  that  will  be  used  by  all.  On-line  advice  on  project 
standards,  instructional  problems,  etc.  is  incorporated  into  the  template  so  all 
developers  can  work  to  the  highest-skill  level  rather  than  the  lowest. 

Each  project  should  begin  with  the  assembling  of  a  project  design  and  standards 
team.  At  this  time  you  should  establish  as  many  of  the  project  ideas  and 
standards  as  you  can.  By  getting  everyone  together  you  have  a  consensus  that 
everyone  buys  into,  helping  to  alleviate  future  problems  of  style  or  instructional 
difference.  This  does  not  eliminate  problems,  but  it  will  help  cut  down  on  them 
significantly.  Once  you  have  a  consensus  on  the  instructional  standards,  the 
templates  and  advice  can  be  developed  or  modified. 

This  is  also  a  good  time  to  begin  establishing  graphics  standards  and  media 
decisions.  The  media  decisions  are  not  whether  or  not  to  use  video  or  audio,  this 
should  have  been  decided  in  a  media  analysis  prior  to  this  point.  However,  how 
much  and  where  to  use  video/audio  can  be  ironed  out.  It  is  important  to  keep  in 
mind  the  relative  costs  of  different  media.  Is  it  cheaper  to  use  a  large  amount  of 
still  video  images,  have  a  graphics  artist  draw  them,  or  scan  photographs  of  the 
images  using  a  color  scanner?  The.se  costs  are  all  known  and  are  obviously  a 
deciding  factor,  but  the  instructional  validity  of  a  given  media  has  to  be 
considered  as  well. 
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It  is  also  important  to  consider  the  “edutainment”  value  for  the  student.  In 
today’s  sophisticated  society,  student’s  are  no  longer  impressed  by  simple 
computer  graphics.  They  are  used  to  seeing  very  advanced  computer  graphics 
and  video  “tricks”  on  television  and  at  the  movies.  This  is  not  to  say  that  the 
expense  of  these  items  gives  you  a  one-to-one  increase  in  training  effectiveness. 
However,  we  would  all  agree  that  given  the  choice  of  seeing  the  same  training 
materials  in  black  and  white  or  in  color  we  would  usually  opt  for  the  color 
presentation,  simply  for  the  increased  visual  appeal.  There  is  a  tightrope  that  . 
must  be  walked  when  making  these  decisions.  The  use  of  256  colors  vs.  16  colors 
may  result  in  more  appealing  graphics  with  greater  shading  options,  etc.  but  the 
cost  of  such  graphics  may  be  so  high  that  you  should  standardize  on  a  16  color 
palette  for  the  majority  of  the  courseware,  leaving  256  colors  for  those  graphics 
where  it  is  essential.  Today’s  technology  allows  you  to  make  such  decisions. 

You  can  also  mix  still  and  motion  video,  computer-generated  as  well  as  digitally 
stored  audio.  Through  the  use  of  DVI  technology  you  can  store  all  of  your  audio 
and  video  information  on  a  hard  disk  or  CD-ROM  and  eliminate  the  use  of  a 
videodisc  player. 

Once  you  have  established  your  instructional,  graphics  and  media  standards  you 
can  design  the  management  and  organization  of  the  project.  The  cost  of  initial 
development  often  contributes  less  than  40%  of  the  life-cycle  cost  of  a  piece  of 
courseware.  In  other  words,  when  you  have  paid  for  the  initial  development  of 
courseware,  you  haven’t  even  spent  half  of  what  you  will  before  you  throw  it 
away.  This  makes  meaningful  organization  and  management  even  more  vital. 

It  is  essential  to  formalize  the  communications  between  each  step  of  the  task  and 
how  data  will  be  gathered  and  stored.  Technology  now  allows  text  and  graphics 
to  be  referenced  in  multiple  lessons  but  stored  in  one  place.  This  aids  greatly  in 
the  development  and  maintenance  tasks.  You  can  store  one  image  of  a  given 
graphic  and  use  it  in  hundreds  of  lessons.  When  that  graphic  image  changes  you 
need  only  change  it  in  one  place  to  have  it  affect  all  lessons.  The  same  is  true  for 
text.  Figure  2  shows  an  example  of  an  aviation  project  that  incorporated  the  use  of 
referenced  graphics  into  the  lesson  generation  process. 
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How  Generation  was  Used  in 
a  Large  Aviation  Project 


Figure  2 


With  this  technology  comes  an  added  responsibility  for  organization  and 
management.  One  tool  that  could  be  invaluable  in  storing  all  this  data  is  an  on¬ 
line  database.  All  text  and  graphics  and  their  related  files  can  be  managed  in  the 
database,  available  for  all  developers  to  retrieve  information  at  any  time.  This 
data  base  could  store  not  only  the  information  about  a  given  graphic  but  could  also 
be  a  dynamic  list  of  all  lessons  that  might  be  affected  by  future  changes.  This 
will  also  increase  overall  productivity  over  the  life  of  the  courseware. 


Technology  will  continue  to  move  forward.  It  is  a  good  idea  to  not  get  bogged 
down  in  the  latest  and  greatest  trends  to  come  along.  They  don’t  always  stick 
around.  At  the  same  time,  it  is  a  good  idea  to  move  forward  with  the  technology. 
Computer  vendors  are  becoming  more  aware  of  the  need  for  flexibility  between 
products  and  standardization  of  capability.  How  many  of  us  own  beta  VCR’s 
instead  of  VHS?  By  careful  analysis  you  can  keep  abreast  of  the  changes  and  take 
advantage  of  them  without  making  costly  mistakes.  The  courseware  development 
task  will  become  increasingly  integrated  in  all  facets. 
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Tlirough  the  use  of  instructional  templates  it  is  possible  to  do  all  initial 
storyboard  development  on-line,  no  longer  relying  on  a  paper-based  world. 

When  you  develop  courseware  using  a  system  such  as  Sage  you  can  try  out  your 
ideas  at  a  much  earlier  point  in  the  process,  without  wasting  valuable  time  and 
money.  The  designer  also  has  the  opportunity  to  incorporate  graphics,  text,  video 
and  audio  that  have  been  under  development  simultaneously  at  a  much  earlier 
point.  This  makes  the  task  of  reviewing  and  revision  much  simpler  and  cheaper, 
because  it  can  happen  earlier  in  the  development  cycle.  Having  one,  integrated 
tool  helps  significantly  with  the  process.  Working  with  a  large  development  staff 
may  require  that  lessons  pass  from  one  developer  to  another  before  completion. 
Having  integrated  development  tools  allows  information  to  be  included  directly 
in  the  file  as  it  is  being  developed,  lessening  the  chance  for  mistakes  to  be  made. 
If  you  have  a  smaller  development  task  where  more  of  the  work  is  done  by  one 
individual  integrated  tools  are  just  as  valuable.  If  you  are  developing  a  prototype 
and  are  relying  on  the  SME  to  develop  the  bulk  of  the  effort,  having  on-line 
instructional  advice,  easy  graphics  storage  and  retrieval  and  simple  audio/video 
development  and  incorporation  allows  him/her  to  develop  a  more  complete 
prototype  without  having  to  rely  so  heavily  on  the  expertise  of  others.  The 
expertise  is  in  the  tools. 

This  is  the  wave  of  the  future.  It  has  to  be.  As  our  workforce  becomes  less 
skilled  relative  to  the  demands  of  the  workplace  and  our  pool  of  talent  shrinks  we 
will  have  to  rely  more  and  more  on  the  technology  and  tools  that  are  being 
developed.  As  trainers  we  will  have  to  become  more  adept  at  our  use  and 
knowledge  of  the  tools  and  technology  that  are  available  and  as  they  are 
developed.  We  can’t  sit  still  because  technology  certainly  doesn’t.  In  order  to 
meet  the  challenges  of  the  future,  we  must  think  and  work  smarter  than  we  have 
before  and  take  frll  advantage  of  the  technology. 
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ABSTRACT 


During  the  past  several  years,  the  benefits  and  risks  of  buying  trainers/training  fi-om  the  aircraft  prime  have  been 
rftgnigcprf  This  paper  provides  a  history  of  the  operator  and  maintenance  trainers  procured  through  the  prime, 
Sikorsky  Aircraft,  for  the  SH-60F  CV  ASW  Inner  Zone  Weapon  System.  The  benefits  and  lessons  learned  from 
this  successful  "Buy  from  the  Prime"  procurement  are  also  described  in  this  paper. 


1  INTRODUCTION 

Igor  Sikorsky  recognized  the  complexity  of  operating 
a  helicopter.  He  built  bis  first  flight  simulator  in  the 
1930’s  as  a  development  and  training  device  for  his 
new  flying  madiine,  the  VS-300.  He  used  the 
simulator  to  study  attitude  control  of  the  main  rotor 
system,  arrangements  of  mechanical  cotttrol 
interfaces,  and  control  gearing  requirements.  Using 


the  simulator,  he  analyzed  several  configurations  of 
flight  controls,  including  tail  mounted  rotors  for  pitch 
arKl  roll  control.  Piloting  techniques  were  developed 
on  the  simulator  before  flight,  as  shown  in  Hgure  1 . 
His  survival  tbrou^  this  early  period  of  helicopter 
development  testifies  to  the  value  of  training  the  pilot 
in  a  simulator  before  flight 
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lACKGEOUND  -  THE  SIS-WF 
AIECRAFT  WEAPONS  SYSTEM 

Under  the  Layeied  Defense  concept,  with  imiEsr, 
middle  and  outer  zottes,  Navy  uses  tite  SH--60B, 
S-3,  atKl  P-3  in  tte  acoustically  more  domiant  waters 
beyow!  the  innsr  zone,  ^prosimstely  30  miles  from 
tte  carrier  battle  group.  In  tte  raucous  inner  area, 
wl^s®  friendly  ships  are  coostantiy  moving,  passive 
sonobuoys  and  tow^  arrays  are  less  effective.  The 
dif^ing  sonar  S^omes  the  piefesrod  ASW  sensor. 
Besides  the  high  ambient  noise,  being  close  to  the 
battle  group  means  that  time  lines  compress.  Miramal 
time  is  available  to  detect,  tradk,  and  release  a 
weapon  against  a  threatening  submarirts.  The 
SH— 60F  was  selected  to  replan  the  SH— 3H  to  patrol 
the  inner  zotse.  The  SH— 3H,  producjsd  in  the  1960’s, 
was  fully  depreciated  aiKl  scarce  when  the  Navy  made 
the  decision  to  upgratte  to  the  SH-60F. 

very  successful  SH-60B  SEAHAWK  design  was 
modrfied  into  the  SH— 60F  for  operation  off  the 
earner,  at  the  center  of  the  inrser  zone.  The  SH-60F 
integrates  the  AN/ASN-150  Tactical 
Data/Communication  System  with  the  AQS— 13F 
Dif^img  Sonar,  sonobuoys  and  MK-46/MK.-50 
torpedoes. 

Two  pilots,  an  Acoustic  Sensor  C^srator  (ASO),  airwt 
a  Tactjcal  SetEsor  Operator  (TSO)  operate  the  mission 
systems.  A  dual  ledundasH  Mtt,-STD-1553B  date 
bus  interfsces  the  saror,  armament,  control  and 
di^lay  subsystems.  The  CV— Helo  is  required!  to  fly 
a  umque  miOTon  in  an  extremely  demanding 
operational  environmers.  Often  at  nighi?^  th® 
CV-H®lo  drqjs  sonobuoys  to  list^  passively  for  an 
aggre^or  sub,  and  thee  uses  «rive  sonobuoy  and 
(lifting  sosw  to  tra^  assl  target  the  threat.  This 
routiraly  requires  Irovering  at  50  feet  at  night  over  a 
hostile  sea,  (%ping  am!  retrieving  up  to  15(K)  feet  of 
sonar  ^mor  rabte,  dashing  to  te  next  dip  point  asHl 
repeatirsg  tl®  pieces.  This  rigowm  mi^on  requires 
hi^y  tiais^  fligta  atid  mainteeanw  crews  to  be 
suc^ssfuL 

Full  scale  development  of  the  aircraft  began  in 
February  1985  and  Navy  ojrerational  evaluation 
(OPEVAL)  WES  concluded  in  February  1988. 
Production  aircraft  (telivery  began  in  May  1989.  The 
Navy  plans  to  procure  ai^roximateiy  175  aircraft.  To 
support  the  training  requirements  of  thi.s  new  we^jons 
system,  the  Navy  selected  Sikorsky  to  provide  trainer 


and  initial  cadre  training. 

3  DESCRIPTION  OF  TRAINING 
DEVICES 

The  Navy  built  two  new  faciMties  to  house  the 
SH-60F  trainers,  classrooms,  aiKl  support  activities. 
These  facilities  ate  located  at  NAS  North  Island, 
California  and  NAS  Jacksonville,  Florida.  Sikoisky 
provided  basic  facility  space,  power,  and  environment 
requirements  to  the  Navy  within  60  days  of  ctmtract 
award.  The  Navy  designed  airi  fabricated  the  first 
facihty  (NAS  NI)  in  less  than  three  years  after 
receiving  the  initial  facility  requirements  document. 

3.1  Matotemaasos  Traia®^  (MT) 

Sikorsky  desigi^d  and  built  Maintenance  Trainers  to 
support  teaching  tte  critical  testing,  trouble  shooting 
and  lemoveAeplace  skills -required  by  orgaitizational 
("O”  Level)  mechanics.  NAVAIR  recomrasemls  tbe 
use  of  Aircraft  Common  Equipmente  (ACE).  Wiring 
harnesses,  black  boxes  and  major  assemblies  (such  as 
gear  boxes  and  rotor  beads)  are  aircraft  common, 
where  possible.  This  approach  (using  aircraft 
common  rather  tharr  trainer  [reculiar  equipment) 
provides  Ite  following  benefits: 

Configuration  identity,  touch,  weight,  and 
feel  of  components  are  the  same  as  <m  tte 
aircraft. 

Noo-recuning  engineering  effort  to 
design  sunulated  comporrents  is 
reduced. 

Spare  par^  are  available  from  aircraft 
common  Navy  inventory. 

MaintenaiEce  tasks,  logistics  data,  and  post¬ 
overhaul  testing  for  te  ACE  are  the  same 
for  aircraft  common  traiirer  compoirents. 

Engineering  changes  to  aircraft  are  more 
readily  incorporated  into  the  trainer. 

The  advantage  in  using  trainer  peculiar  equipment  Ls 
that  malfunctions  and  trainer  unique  requirements  are 
more  easily  designed  into  the  system.  Using  aircraft 
common  equipment  Limits  malfunctions  to  being 
inserted  by  outside-box  stimulus  or  software. 
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Figure  2  Composite  Maintenance  Trainer 


The  Maintenance  Trainers’  design  concepts  were 
based  on  the  SH-60B  Maintenance  Trainers,  which 
were  delivered  to  Mayport  and  North  Island  in  the 
late  1980s. 

The  Navy  facilities  at  NAS  North  Island  and  NAS 
Jacksonville  have  the  SH-60F  Maintenance  Trainers 
listed  in  Table  I. 

3,2  OPERATOR  TRAINERS 

The  design  concepts  for  the  Operator  Trainers  were 
developed  by  the  Navy  and  defined  by  the  Navy’s 
Design  Specifications.  These  specifications  were  the 
basis  for  a  formal  competitive  bidding  process  run  by 
Sikorsky.  The  successful  bidders  were  Norden 
Systems  Company  for  the  Weapon  System  Trainer 
(WST)  and  Acoustic  Trainer  (AT),  and  Teledyne 
Systems  Company  for  the  Tactical  Team  Trainer. 


Reflectone,  Inc.  was  the  principal  subcontractor  to 
Norden  and  provided  the  motion  base,  aero  model, 
instructor  stations,  and  some  systems  integration 
effort.  Norden  provided  the  Acoustic  Trainers, 
Sensor  Operator  Trainers,  underwater  acoustic 
simulation,  and  Tactical  Interface  Unit.  The  Operator 
Trainers  are  listed  in  Table  U. 

The  Operator  Trainer  design  criteria  included  the 
following: 

New  designs  -  per  Mil-Specification  (for 
hardware  and  software) 

Commercial  equipment  -  permitted  where 
economically  appropriate  (example:  Visual 
System) 
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TaW®  I  Maifflteaian^  Trainefs 


RAST/TaO  Wbeel/HoM  MT 


Acoustic  data  -  Navy’s  Common  Acoustic 
Data  Base  (CADB)  and  ocean  data  were 
used  as  source  data  for  underwater  targets  in 
their  environment 

Aircraft  common  equipment  -  when  used,  are 
flight  qualified  and  interchangeable  with  the 
aircraft. 

Note:  Sikorsky  modified  the 

aircraft  common  Automatic  Flight  Control 
Computer  to  allow  its  use  in  either  an 
aircraft  or  in  a  trainer  (where  problem  freeze 
is  required). 


Government  Furnished  Equipmertt  (GFE) 
none. 


Figure  3  Operational  Flight  Trainers 


ACOUSTIC  trainer  (AT) 
SO'  •  SfNSOfl  OPEOAfOR  trainer 

OfT  •  operational  EliGhT  trainer 


Figure  4  Coupled  Modes  and  Mission  Modes 


4  INTEGRATED  LOGISTICS  and 
RELIABILITY 

The  Navy  has  a  support  contract  with  Loral  for  on¬ 
site  maintenance  of  the  Maintenance  and  Operator 
Trainers  to  ensure  that  they  are  frilly  operational. 
NAVAIR  procured  a  "seed"  set  of  spare  parts  from  a 
vaiiety  of  sources  to  support  the  trainers.  Loral  is 
required  to  repair  or  replace  any  equipment  as 
necessary.  The  reliability  experience  at  NAS  NI  has 
been  very  good.  During  the  past  year,  the  trainers 
have  been  available  for  scheduled  training  more  than 
95%  of  the  time. 

5  TRAINING  OF  INSTRUCTOR  CADRE 

Sikorsky  provided  a  Pilot  Familiarization  course  that 
described  the  difrerences  between  SH-frOB  and 
SH-60F  aircraft.  This  course  was  given  to  SH-60B 
qualified  pilots,  who  became  instructor  pilots  at 
HS-10.  Sikorsky,  Norden  aixi  Reflectone  |»esented 
instructor  utilization  courses  to  pilots  and  sensor 
operators.  These  courses  provided  detailed  instruction 
on  how  to  qierate  the  trainers. 

6  "BOUGHT  FROM  THE  PRIME" 
ADVANTAGES 

There  are  several  benefits  to  the  customer  in 
purchasing  Trainers  and  Training  from  the  aircraft 
prime.  These  advantages  are  most  viable  when  the 
trainers  support  a  new  model  aircraft  or  a  significant 
Engineering  Change  Proposal  (ECT),  which  are  being 
developed  by  the  prime.  These  advantages  are 
described  below: 

6.1  Single  Point  of  Contact  or  "One  call  does 
it  aO." 

A  single  point  of  contact  saves  cost  and  schedule  to 
the  customer  for  essentiaUy  all  the  routine  and 
abiwrmal  problems  and  tides  associated  with  any 
major  trainer  procurement.  The  customer  has  one 
discussion  point  for  the  innumerable  issues  and 
decisions  required.  With  a  multiple  set  of  suppliers 
and  data  sources,  the  customer’s  effort  to 
communicate  aiKl  coordinate  issues  and  answers 
would  be  significant. 
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(TTT) 

Active  Tactical  Data  System 

(2  aircraft  per  TTT) 

InsfractOT  plays  Sensor  Operators  roles 

T^tical  Environment  simulation 

2  aircraft  may  be  indejrendent  or  integrated  in  1  problem 

world 

Since  trainer  (tesigm  shomid  use  aircrafi  commoo 
equipment  (ACE)  wterg  po^bls,  tte  Prim®  provides 
a  single  authority  to  spmfy,  qualify,  prooiro,  test, 
accept  and  allocate  the  supplies. 

Data  Transfer:  Use  prime  has  the  respomibility  to 
fulfill  tte  aircraft  requirements  and  integrate  aircraft 
component  suppliers  into  a  system.  Tl^refore,  the 


prime  noimaily  writes  tire  compoEent  requirement 
specifications,  interface  requirements,  and  acceptance 
criteria.  This  information  is  critical  to  the  trainer 
design  engineers.  It  is  essential  for  accurately 
incorporating  these  components  into  tire  training 
environment.  Using  a  thiid  party  trainer  supplier 
would  require  the  customer  to  be  responsible  for  and 
to  orchestrate  transfer  of  data  between  aircraft 
component  supplier  and  trainer  supplier.  This 
requires  the  Customer  to  promote  Non-Disclosure 
Agreements  (NDA)  between  the  two  companies.  The 


71 


prime  motivates  each  supplier  to  enter  into  NDAs  and 
takes  responsibility  for  accuracy  and  timeliness  of  the 
data. 

Procurement  of  Aircraft  Common  Equipment  (ACE): 
As  the  auciaft  manufacturer,  the  prime  is  required  to 
contract  with  suppliers  to  provide  aircraft  components. 
The  CV-Helo  trainer  manufacturers  used  many 
aircraft  common  parts.  It  was  relatively  easy  for  the 
trainer  parts  to  be  "coat  tailed"  on  existing  ACTE 
purchase  orders.  This  provided  the  following 
advantages: 

Economy  of  scale  (lower  cost  with  larger 
volume). 

Reduced  time  to  implement  a  Purchase 
Order  with  ACE  siqiplier.  (Do  not  need  to 
competitively  bid  small  quantity  order, 
negotiate  cost,  schedule  and  Terms  and 
Conditions). 

The  prime  may  be  aUe  to  reallocate  critical 
assets. 

Rilly  qualified  ACE  used  on  the  trainer  is 
accepted  using  identical  accqitanoe  criteria 
as  the  aircraft.  The  configuration  identities 
(part  number  and  software  load)  are  identical 
with  the  aircraft. 

In  the  dual  role  of  aircraft  prime  and  trainer  prime, 
Sikorsky  understood  the  significant  benefit  to  the 
customer  in  making  the  Automatic  Flight  Control 
System  Cmnputer  (AFCSC)  common  fw  aircraft  and 
trainer  q^lications.  Historically,  spedal  software  had 
to  be  incorporated  into  the  AFCSC  to  use  it  in  a 
trainer.  With  the  full  coqreration  of  the  aircraft 
engineers,  the  SH-60F  AFCSC  was  redesigned  to  be 
both  aircraft  and  Trainer  compatiUe.  The  TraitMr 
requires  Trainer  fieeze”,  restart,  reinitialize  normally, 
and  reinitialize  to  a  qredal  "Initial  condition".  The 
AFCSC  itKorporated  these  ^redal  effects  to  allow  its 
use  in  the  Trainer  and  in  the  aircraft  without  Trainer 
Peculiar  Software.  The  AFCSC  automatically  senses 
its  environment  through  multiple  redundant  software 
atxl  hardware  sensors. 

In  some  historical  situations,  the  Trainer  Program 
Manager  has  lost  assets  to  support  aircraft  delivery. 
On  the  CV-Helo  program,  the  opposite  was  the  case. 
Aircraft  Common  Equipment  was  borrowed  from 


aircraft  aUocations  (on  the  same  contract  as  the 
trainers),  and  given  to  the  trainer.  This  reallocation 
was  a  critical  factor  in  delivering  NAS  JAX  trainers 
on  schedule. 

63  Commonality  of  Parts  (Aircraft  and 
Trainer)  Minimizes  Inventory  Spares  and 
Logistics  Data 

Use  of  aircraft  common  equipment  in  the  trainer 
configuration  allows  the  customer  to  have  a  single 
inventory  item,  rather  than  one  for  each  application 
(trainer  and  aircraft).  Non-recurring  design  costs  for 
the  trainer  umque  (simulated  item)  are  not  required. 
The  life  cycle  costs  associated  with  supporting  one 
rather  than  two  sets  of  inventory  items  is  significant. 
Common  software  configurations  and  common  data 
items  are  major  cost  minimizing  drivers.  One  group 
of  I-level  and  D-level  data,  tooling,  aixi  test  sets  is 
required.  Navy  inventory  from  the  trainer  can  be 
exchanged  with  components  from  aircraft  stores 
during  maintenance.  The  trainer  ACTE  can  be  repaired 
and  retunied  to  aircraft  spares  stock. 

6.4  Aircraft  Engineer  to  Trainer  Engineer 
Communication 

The  trainer  design  engineer  typically  (and 
appropriately)  desires  much  more  data  describing  the 
aircraft  system  than  is  available.  To  minimize  the 
filtering  of  information,  the  trainer  suppliers  were 
asked  to  identify  any  aircraft  data  that  they  required 
before  the  trainer  contract  was  signed. 

Early  in  the  program,  the  trainer  engineers  visited  the 
SH-60F  Haidware/Software  Integration  Facility 
(HSIF).  The  HSIF  provides  a  hot  bench  for  testing 
tactical  data  system  components.  The  HSIF  provided 
the  trainer  engineers  with  their  first  live  demonstration 
of  the  Tactical  Data  System. 

Aircraft  Engineering  Change  Proposals  are  routinely 
routed  to  the  prime’s  Trainer  Engineering  to 
determine  applicability  of  the  E(3P  on  the  trainers. 
Many  ECT»s  are  not  training  significant.  Some  ECPs 
are  recommended  for  incorporation  into  the  trainer. 
The  close  ties  between  aircraft  and  trainer  engineers 
mmimize  the  time  to  incorporate  ECPs  into  the 
trainers. 
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6^  Acc@p4afflsss  CriStsrna 

A  critical  c<®cein  for  any  Of^rator  Trainer 
manufacturer  and  user  is  the  acceptance  criteria. 
There  have  been  oumerous  instances  when  trainer 
deliveries  have  been  delayed  because  the  acceptance 
criteria  was  in  the  hamls  of  the  acceptance  pilot 
(literally).  Conuisumcating  what  tte  pilot  fitsds 
acceptable  (what  feels  rigjjt)  to  the  design  engineer 
can  be  an  aiduars,  time  ctMssuming  effort  One  of  the 
more  esoteric  areas  is  in  the  Digital  Control  Loading 
requirements.  Digital  Control  Loading  employs  a 
hydro-mechanical-software  system  to  simulate  the 
coirtrol  loads  on  the  pilots’  tends  am!  feet.  The 
for^/position/velocity  espsrierEced  by  tte  pilot  in  the 
corarols  provides  significant  trainieg  cues  to  the 
student  and  is  a  cirflenge  to  the  design  engkceer. 

In  onfer  to  provide  early  definition  of  tte  conliol 
loading  reqtairemente,  Sikorsky  provited  an  airmft  to 
tte  trairter  design  engineers,  who  ctdlected  tte  data. 
This  data  was  then  made  part  of  tte  criteria  for 
accqjtance.  During  impeoion,  tlm  data  provided 
discrete  acceptanra  criteria  for  tte  cmstrol  loading 
system. 

Becar^  tte  SH“60F  aircraft  is  a  derivative  of  tte 
SH— 60B  and  UH-60A  (BlaAhawk),  little  flight  test 
data  was  collect^  for  tte  SH-60F.  Sikorsky  had 
collected  significartt  data  on  tte  predecessor  aircraft 
Through  close  correlation  between  the  Navy  test 
pilots,  Sikorsky  test  pilo^  and  aero  modeling 
engineers,  and  Refiectotre  engineers,  tte  historical 
data  was  analyzed  in  depth.  From  this  large  amount 
of  flight  test  data,  tte  traitrer  srero  model  criteria  wss 
defined  early  in  tte  program. 

Use  above  aero  ami  control  losing  souir%  data  was 
m^fe  available  to  tte  trainer  dsrigsrets  ^uiy  in  tte 
program.  Signifiauri  cost  and  sdiedule  drivers  were 
avrated  because  this  dara  was  available  early  in  tte 
program  ami  pritH-  to  first  flight  of  prototype  aircraft, 

6,6  Mimlmiz®  Government  Furnished 
Equlpmma  (GFE)  and  Government 
FianaMsed  Informatiom  (GFl) 

Otre  of  fire  most  often  used  reasons  for  contractor 
excusable  delivery  delay  is  late  GFE  and/or  late  GFI. 
By  employing  the  prime,  GFE,  including  Support 
Equipment,  requirements  were  reduced  to  zero.  No 
GFE  was  provided  under  tte  contract  for  CV-Helo 


trainers.  By  employing  the  prime’s  aircraft  data 
configuration  definition,  aircraft  handling  data,  ami 
tte  aircraft  itself,  GFI  requirements  were  reduced  to 
classified  submarme  target  data,  ocean  tem;rerature 
curves,  and  some  visual  scenes  data.  Thus,  customer 
risk  is  greatly  reduced  while  design  information  is 
provided  wten  ireeded. 

6.7  Employ  Aircraft  Toolmf 

Tte  hardware  Maintenance  Trainsis,  such  as  tte 
Composite  ami  Avionics  MTs,  employ  a  large  number 
of  aircraft  common  parts.  Parts  such  as  tte  aircraft 
upper  deck  and  fuselage  components  are  assembled  in 
tte  same  final  assembly  production  line  as  tte 
aircraft.  Tooling  required  for  the  aircraft  is  used  for 
tte  trainer.  To  fmalitate  production  control  and 
plarming,  each  trainer  in  the  assembly  line  is  assipied 
a  dmmy  aircraft  tail  number. 

6  J  Trairsifflg  Begisss  Early  in  the  Program 

Tte  delivery  of  trainers  allowed  qualification  ami 
training  of  pilots  and  maintainers  in  a  timely  manner 
to  support  HS-10  readiiress.  Ttese  trainers  reflected 
an  aircraft  configurarion  wtsich  was  only  a  few 
montte  older  than  tte  configuration  of  tte  fielded 
fl^t  aircraft.  This  concept  aOows  tte  trainer  to  be 
on-site  in  time  to  begin  training  much  earlier  than  if 
tte  trainer  design  had  been  delayed  to  reflect  later 
aircraft  configurations.  A  trainer  upgrade  is  underway 
to  bring  trainer  configuration  up  to  tte  current  fleet 
aircraft  configuration. 

7  "BOUGHT  FROM  THE  PRIME" 
ADVANTAGES  DECLINE  WHEN«,. 

7.1  Ahncraft  Design  is  Matnr@ 

A  mature  aircraft  design  me^  ttet  data  describing 
tte  aircraft  configuration  is  stable  and  re^lily 
available  to  any  interested  party.  Under  this 
condition,  tte  communication  link  tetw^n  aircraft 
designer  aitd  traitrer  designer  is  no  longer  vital  to 
meeting  a  tight  schedule. 

13.  GFE/GFI  is  Readily  Available 

When  aircraft  common  equipment,  i.e.  GFE,  and 
Government  Furnished  Information  (GFI)  are  readily 
available  frmn  tte  Government,  the  benefit  of  using 
the  prime’s  pool  of  equipment  and  data  declines. 
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73  Simulated  Not  Stimulated  Aircraft 
Equipment  Used  in  Trainer 

Tbe  design  philosophy  of  employing  Simulated 
Aircraft  Equipmem  rather  than  Stimulated  (Aircraft 
Common)  Equipment  removes  the  prime’s 
coraribution  of  providing  a  readily  available  pool  of 
aircraft  boxes.  Simulated  equipment  provide  the 
following  trade  offe  associated  with  aircraft 
equipment: 

Increases  Non-Recurring  Costs 

Decreases  Recurring  Costs 

Facilitates  simulated  malftincrion  insertion 
into  tbe  box 

Significantly  drives  iq)  cost  and  !crh(»ditl<» 
required  to  reflect  aircraft  ECP  (Engiiieering 
Change  Proposal) 

8  COST  SAVINGS 

The  following,  which  were  discussed  above,  can  be 
considered  in  the  cost  savings  equation  for  buying 
ftom  tbe  prime; 

Program  and  Inventory  cost  of  new  aircraft 
waiting  for  trained  personnel  is  minimiwH 
Major  delays  in  training  commencement  may 
jeopardize  the  aircraft  program,  as  well  as 
the  trainer  program. 

Use  of  Aircraft  Common  Equqxnent  in  the 
trainers  minimizes  costs  of  tte  ftdlowing: 

Initial  box  procurement 

Non-recurring  Engineering 

Inventory 

Maintenance 

Initial  and  life  cycle  data 

Upgrade  (ECPs) 

Customer  management  in  securing  GFE  and  GFI. 

Supplier  compensation  for  delayed  GFE  and 
GFI. 

Customer  management  of  ECP  incorporation. 


Providing  trained  crews  and  maintaineis 
early  in  tbe  life  of  a  new  we^ns  system. 
(This  is  difficult  to  quantify,  but  a  major 
consideration.) 

Tbe  above  cost  savings  outweigh  the  perceived  extra 
costs  (overhead)  that  may  be  encountered  in  procuring 
third  party  trainers  through  the  prime. 

9  LESSONS  LEARNED 

9.1  Background:  Several  changes  to  the  aircraft 
software  were  made  subsequent  to  Trainer  PDR  and 
CDR.  Trainer  delivery  was  delayed  several  months 
in  mder  to  incorporate  these  aircraft  changes. 

Recommerxlation:  When  trainers  support  newly 
develtqred  aircraft,  tbe  contract  should  provide  a 
vehicle  to  allow  for  a  Block  Upgrade  after  trainer 
delivery.  The  Block  Upgrade  would  best  be  served 
with  a  cost  plus  type  contract  because  tbe  extent  of 
the  Block  Upgrade  is  impossiUe  to  determine  several 
years  in  advaixte. 

9.2  Background;  Simulated  aircraft  compoitents 
were  permitted  to  be  employed  in  the  Operator 
Trainers.  This  is  appnqniate  because  some 
instruments  (such  as  tbe  air  pressure  driven  airspeed 
indicator)  are  quite  expensive  to  stimulate,  lliese 
instruments  also  have  mature  configuration  definition. 
Simulated  components  which  are  likely  catxlidates  for 
aircraft  Engineering  (Tbange  Pttqrosals  (ECP)  requite 
significant  ix>n-iecutring  engineeting  when  the  ECPs 
are  itKorporated  into  the  trairxr.  These  components, 
such  as  tbe  Armament  System  Controller  (ASC),  are 
complicated  to  simulate  arxl  ate  schedule  drivers 
daring  traiiKr  upgrades. 

Recommendation:  Simulated  irstrurttents  should  be 
limited  to  those  which  are  unlikely  ECP 
Aircraft  common  equipment,  such  as  the  ASC  and 
Commumcations  System  Controller  (CSC),  should  be 
employed  in  tbe  trainer.  This  will  facilitate 

incorporation  of  aircraft  ECPs. 

10  CONCLUSION 

Where  training  of  aircraft  crews  and  maintaineis  is 
required  soon  after  delivery  of  a  ikw  aircraft  weapons 
system,  there  are  significant  cost  aixi  schedule 
benefits  in  procuring  the  training  devices  ftom  the 
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aircraft  prime.  Commamcation  debys  {^rwesn  te 
aircraft  (tesigner  am!  the  traiEsr  (fesigtser  asie 
minimized.  The  i^ed  for  GRE  /  and  most  GFI  can  be 
avoided  becaase  of  the  prime’s  inherent  custody  and 
interest  in  this  material  /  data.  Given  that  aircraft 
commoQ  equipment  (ACE)  (rater  than  simulated 
equipment)  provicfe  the  design  ^proach  for 
trainers,  the  prime  is  l^st  jwstesed  to  providfi  ACE. 
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ABSTRACT 

In  this  paper?  we  propose  to  incorporate  elements  of  education  and 
training  in  a  seamless  architecture  that  would  promote  enhanced 
training  effectiveness  in  this  era  of  shrinking  budgets^  Specifically? 
we  will  discuss  i  nterdependenc  i  es  in  the  methods  used  in  the  classroom 
educational  environment  and  the  hands-on  training  environment.  We 
will  examine  how  analogous  functions  of  the  educational  and  training 
environments  can  be  integrated  to  facilitate  seamless  training  in  a 
cost-effective  manner.  For  example?  we  will  discuss  how  the 
instructor  console  in  the  educational  world  may  be  combined  with  the 
capabilities  of  the  master  control  console  and  command  centers  in  the 
training  world?  and  how  the  brief  and  debrief  capabilities  of  the 
training  world  can  provide  valid  simulation  to  review  lessons  learned 
in  the  classroom  world.  In  bridging  educational  and  training 
environments?  we  will  identify  technologies?  such  as  networking  and 
artificial  intelligence?  that  can  enhance  training  effectiveness 
within  those  environments.  We  will  also  discuss  future  research 
directions  to  address  the  overall  training  problem  of  merging 
educational  and  training  environments, 

INTRODUCTION 

Typically?  the  training  environment  consists  of  a  sequence  of  discrete 
activities  (see  Figure  1):  (1)  classroom  lectures?  (2)  hands-on  training 

in  a  lab  containing  a  simulator?  and  (3)  hands-on  training  with  the 
actual  tactical  system  through  field  exercises  or  embedded  training. 

The  classroom  environment  is  a  typical  lecture  environment  that  is 
focused  on  instructional  presentations  using  media  devices  such  as  an 
overhead  projector?  a  VCR?  and  a  blackboard.  The  lectures  are 
reinforced  by  hands-on  training  in  a  lab  containing  simulators  that 
typically  do  not  completely  emulate  the  tactical  system.  To  augment 
classroom  lab  exercises?  debrief  activities  are?  then?  used  to  highlight 
and  reinforce  the  devised  training  objectives.  Following  these 
schoolhouse  activities?  skill  development  continues  with  on-the-job 
training  through  field  exercises  and  normal  daily  use. 

Presently?  the  bulk  of  training  is  done  using  the  real  tactical  system? 
whether  it  be  from  normal  daily  use?  planned  field  exercises?  or  use 
of  embedded  training  capabilities.  Future  efforts  must  concentrate 
on  providing  training  improvements  that  will  move  the  bulk  of  training 
into  the  classroom  and  simulator  lab  and  will  result  in  overall  savings 
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Figure  1.  Traditional  Training  Environment 


to  the  Department  of  Defense  (DoD)  by  reducing  usage  of  the  tactical 
hardware.  Our  approach  will  form  a  comprehensive  training 
environment  that  will  not  only  enhance  training*  but  also  facilitate 
access  to  training  by  a  greater  number  of  individuals. 

To  achieve  the  stated  objective,  an  integrated  structure  (see  Figure 
3)  will  be  explored  for  training  groups  (classroom),  individuals 
(computei — based  self-instruction),  teams  (team  trainers),  and  groups  of 
teams  (simulated  battlefield,  which  can  include  land,  air,  and  sea 
forces).  To  transfer  a  greater  percentage  of  training  activities  to 
the  simulator  world,  each  trainer  requires  realistic  simulations, 
connectivity  to  a  network  of  trainers  for  simulated  field  exercises, 
and  training  curricula  ranging  from  the  individual  level  up  to  the 
highest  field  command  level  in  each  of  the  four  activities  above. 
Critical  to  the  cost-effective  seamless  training  environment  solution 
is  the  ability  for  easy  movement  into  and  out  of  each  training 
activity  through  common  features  and  to  easily  access  each  activity 
through  networking  remote  locations  and  low-cost  individual  training 
devices 

In  the  following  sections,  we  will  discuss  conceptual  requirements  for 
the  proposed  integrated  training  environment  that  entails  classroom 
training,  computer-based  autodidactic  (self-instruction)  training,  team 
training,  and  battlefield  training.  For  each  training  activity,  we  will 
identify  existing  technologies  that  are  needed  to  address  the 
conceptual  requirements.  We  will  discuss  the  critical  role  of 
artificial  intelligence  and  networking  as  two  areas  of  technology,  that 
are  required  to  enhance  training  effectiveness.  We  will  conclude  with 


78 


I 


Figure  So  New  Training  Environment 


some  remarks  regarding  future  directions  for  training  systems 
development.  No  DoD  standards  discussions  are  included?  but 
standards  like  those  for  operating  systems  (POSIX?  Portable  Operating 
System  Interface  for  Computer  Environments)?  interactive  courseware 
(ICU9  Interactive  Courseware)?  logistics  (CALS?  Computer-aided 
Acquisition  and  Logistics  Support)?  and  communications  (BOSIPs 
Government’s  Open  System  Interconnect  Profile)  are  expected  to  be 
used  for  these  existing  areas  and  to  be  established  for  new  elements 
of  the  integrated  training  environment, 

TRAINING  ENVIRONMENT  CONCEPTS 

Conceptual  requirements  for  the  proposed  integrated  training 
environment  are  addressed  by  the  four  training  types  as  identified 
above;  Classroom  Training  (groups)?  Computer-Based  Autodidactic 
Training  (individuals)?  Team  Training  teams)?  and  simulated  Battlefield 
Train! ng  (groups  of  teams  from  land?  sea  ?  and  air)  ,  The  integrated 
training  environment  requirements  are  intended  to  provide  common 
structures  and  tools  across  the  four  training  activities?  to  improve 
access  to  training  for  individuals  and  groups?  and  to  increase  training 
realism.  In  addition,  such  an  integration  environment  provides 
hands-on  training  starting  in  the  classroom?  and  provides  connectivity 
to  allow  large-scale  networked  battlefield  simulations, 

CLASSROOM  TRAINING 

A  requirement  to  incorporate  learning  systems  into  the  standard 
classroom  provides  a  foundation  for  training  improvements.  The 
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purpose  of  learning  systems  is  to  enhance  instructor  authoring  and 
delivery  using  the  integration  of  various  medias.  The  basic  elements 
for  learning  systems  include:  (1)  training  analysis  and  courseware 
design;  (2)  courseware  development*  delivery,  and  evaluation;  (3) 
training  management;  and  (4)  communication.  Current  learning  systems 
provide  instructors  with  efficient  curriculum  design  and  development, 
and  easy  access  to  text,  graphics,  audio,  animation,  and  full  motion 
video.  Learning  systems  also  provide  interaction  with  the  student, 
control  of  classroom  information,  access  to  available  resources,  and 
communications  among  all  personnel  involved  in  a  training  system. 
Enhancing  the  learning  systems  to  allow  integration  of  tactical  system 
simulations  provides  a  foundation  for  hands-on  training  in  the 
classroom.  Classroom  Training  concepts  intended  to  meet  these  needs 
are  discussed  in  the  following  sections: 

INSTRUCTOR  PRESENTATION  CENTER:  An  Instructor  Presentation  Center 
(IPC)  is  a  device  that  integrates  various  types  of  media  under 
computer  control.  It  provides  an  i ns true  tor  with  the  “ability  to  smoo  th  1  y 
present  material  from  vastly  different  media  including  overhead  foils; 
excerpts  from  books,  magazines,  and  papers;  videotapes;  simulations  of 
the  tactical  system;  audio  tapes;  laser  discs;  computer-generated 
displays;  student  inputs;  and  more.  The  IPC  contains  controls  to 
present  material,  and  to  communicate  with  student  stations  and  remote 
training  sites. 

SIMULATION:  The  classroom  includes  the  integration  of  simulations 
from  the  lab  team  trainer  into  the  classroom  presentations  using  an 
inexpensive  hardware  platform.  The  simulations  are  responsive  and 
interactive,  providing  hands-on  training  in  the  classroom. 

STUDENT  STATION  ( SS ) :  The  classroom  includes  student  stations  that 
provide  a  full  spectrum  of  interactive  exercises  ranging  from  high 
fidelity  simulations  to  true/false  guestions.  The  classroom  student 
station  and  the  device  for  Computer-Based  Autodidactic  Training  share 
the  same  functionality  within  the  integrated  training  environment. 

NETWORKING:  The  IPC  and  the  student  stations  are  networked  to 
provide  the  instructor  the  capability  to  activate  an  exercise 
simultaneously  at  each  student  station  and  collect  responses  from 
each  student  station.  Provisions  for  student  stations  to  share  data 
are  included  to  allow  interactive  training  between  multiple  stations. 
Reduced  training  travel  costs  and  improved  access  to  training  is 
accomplished  by  extending  the  classroom  activities  to  remote  locatioris 
through  satellite  or  terrestrial  networks.  The  classroom  IPC  is 
networked  with  the  Team  Training  and  Battlefield  Training  control 
centers  for  debrief  activities. 

ARTIFICIAL  INTELLIGENCE  (AI):  The  IPC  provides  the  ability  to  collect 
student  responses  to  exercises,  analyze  the  responses,  prepare 
statistical  summaries  on  performance  for  feedback  to  students,  and 
even  allow  simulated  instructor  feedback  during  class  exercises  to 
provide  individualized  computei  based  instructional  interaction  that 
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can  supplement  the  actions  of  an  instructor.  For  simulation 
presentations?  various  controls  such  as  target  movement  and  reaction 
characteristics  are  automat i ca 1 1 y  controlled  by  AI  algorithms.  A1 
algorithms  are  exploited  wherever  instructor  workload  can  be  reduced. 

DEBRIEFING  CAPABILITYs  The  IPC  is  capable  of  presenting  debrief 
material  following  Team  Training  and  simulated  Battlefield  Training 
exercises.  The  IPC  and  the  Team  Training  and  Battlefield  Training 
control  centers  are  connected  to  allow  debrief  materials  to  originate 
from  each  location. 

COMPUTER-BASED  AUTODIDACTIC  TRAINING  (CBAT) 

CBAT  provides  an  alternative  to  traditional  classroom  and  lab  training 
methods.  Individuals  can  use  CBAT  to  foster  se  1  f -deve 1 opment  outside 
the  classroom  and  group  training  activities.  This  training  can  occur 
at  a  schoolhouse  or  a  remote  location?  or  while  statiiDned  on  a  large 
weapons  platform.  The  requirements  of  CBAT  include  the  following: 

HARDWARE  ARCHITECTURE:  The  CBAT  host  hardware  supports  a  variety 
of  training  modes  including  basic  operator  training?  interactive 
operator  training?  team  training?  tactics  training?  lecture 
presentation?  and  simulated  battlefield  training.  The  CBAT  device  is 
operational  in  a  standalone?  group?  or  classroom  situation.  It  allows 
interconnection  to  other  devices?  and  provides  interaction  with  audio 
and  video  presentations.  PCs  or  workstations  offer  a  cost-effective 
solution  that  is  easily  transportable  and  expandable. 

SOFTWARE  ARCHITECTURES  The  CBAT  device  can  host  a  variety  of 
software  applications?  both  real-time  and  non— real-t ime ?  that  exhibit 
modularity  and  portability  to  allow  maximum  reuse.  A  menu-driven 
interactive  environment  is  provided  to  allow  the  user  to  select  from  a 
variety  of  training  modes  and  specific  training  scenarios. 

SIMULATION:  The  CBAT  device  can  run  real-time  simulations  with 
display  presentations  as  contained  in  the  Team  Trainer  simulator . 

This  gives  an  individual  student  the  ability  to  develop  and  polish 
skills  outside  the  classroom  or  team  training  activities, 

NETWORKING:  The  CBAT  devices  can  communicate  together  in  a 
classroom  configuration  or  in  groups  that  would  operate  as  a  subset 
of  the  tactical  system.  This  supports  both  vertical  communications  to 
command  centers  and  horizontal  communications  to  peer  simulators  in  a 
team  as  part  of  a  distributed  interactive  s i mu  1  a t i on . C 1  1 

ARTIFICIAL  INTELLIGENCE;  The  CBAT  device  provides  instructional 
feedback  to  the  student.  This  is  achieved  with  AI  algorithms  that  act 
as  an  instructor  by  providing  instructions?  performance  evaluation? 
and  feedback  to  the  student.  The  control  of  scenario  parameters 
(that  is?  Semi-Automated  Forces  or  SAFOR )  in  any  simulation  application 
are  under  AI  control  to  release  the  student  from  activities  that  are 
not  specifically  related  to  the  lesson  goals.  Interactive  algorithms  to 
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simulate  supervisor  and  peer  responses  to  the  student’s 

communications  allow  communications  skill  development  with  supervisors 
and  peers. 

LECTURE  REVIEW;  The  CBAT  device  supports  the  incorporation  of  an 
"encyclopedia"  of  information  for  student  retrieval.  The  information 
consists  of  text,  video,  and  audio  data  integrated  into  recorded 
lectures  and  reference  information  which  the  student  can  recall. 

TEAM  TRAINING 

Team  Training  encompasses  training  on  tactical  system  simulators  that 
involve  a  group  of  individuals  assigned  to  a  single  weapons  system 
platform,  such  as  a  tank,  a  helicopter,  or  a  submarine.  The 
requirements  for  Team  Training  include  the  following: 

HIGH  FIDELITY  SIMULATION:  The  fidelity  of  each  simulator  is  improved 
towards  realism.  The  simulation  of  the  env i r onment ,"  veh i c 1 es  ,  and 
other  objects  and  the  more  complete  emulation  of  the  tactical  system 
are  r equ i r ed . 

ARTIFICIAL  INTELLIGENCE:  Each  simulator  includes  expanded  SAFOR 
capabilities  with  each  simulated  force,  both  friendly  and  unfriendly, 
having  capabilities  such  as  characteristic  motion,  detection,  and 
decision  making.  The  ability  to  monitor  and  analyze  student 
performance  is  accomplished  by  the  use  of  maturing  AI  technologies. 

NETWORKING;  The  simulation  architecture  for  future  team  trainers 
consists  of  a  group  of  processors  in  a  distributed  architecture  in 
which  communications  between  processors,  that  contain  functional 
components  of  the  entire  platform,  are  established  to  provide 
real-time  interactive  simulation.  This  distributed  architecture  is 
characterized  by  a  high  degree  of  modularity  and  portability  that 
allows  for  ease  in  system  extensibility-  Each  team  training  simulator 
is  capable  of  communicating  with  other  team  trainers  to  allow 
mu  1 1 i -p 1 atf orm  simulated  field  activities.  Passive  connections  to  a 
mu  1 1 i - tr a i ner  training  session  is  permitted  to  allow  a  site  to  train 
separately  using  the  field  exercise  scenario  and  events.  In  addition, 
debrief  activities  are  supported  by  interconnecting  the  simulator  with 
the  classroom  IPC. 

BATTLEFIELD  TRAINING  ( BT ) 

The  objective  of  Battlefield  Training  is  to  provide  an  integrated 
environment  for  collective  training  at  different  organizational 
levels — such  as  platoon,  company,  and  ba 1 1 a  1 i on- 1 eve  1  units — to 
conduct  tactical  engagements  in  a  combined  arms  battle  simulation. 

The  battlefield  environment  supports  large-scale  networking  of  a 
massive  number  of  interactive  manned  simulators  to  participate  in  a 
war-gaming  exercise.  Additionally,  Battlefield  Training  supports  a 
viable  semi -automated  forces  simulation  that  is  capable  of  generating 
both  friendly  and  opposing  forces  in  a  complex  and  realistic 
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operational  setting  using  artificial  intelligence.  The  requirements 
for  Battlefield  Training  include  the  following; 

BATTLEFIELD  COMMAND  CENTER;  Battlefield  Training  is  controlled  from 
a  command  center  that  manages  the  simulation  and  identifies  all 
simulator  platforms  par t i c i pa t i ng  in  a  given  exercises  including  those 
of  SAFORo  The  command  center  communicates  appropriate  SAFOR  and 
environmental  scenario  events  in  real-time  to  each  simulator  in  the 
training  session  with  a  common  protocol*  such  as  the  DIS  protocol. 
Similarly*  each  participating  simulator  communicates  scenario  events 
for  its  platform  to  the  command  center  and  other  participants.  The 
command  center  can  conduct  more  than  one  battlefield  training  session 
simultaneously.  Individual  trainers  can  be  upgraded  with  command 
center  capabilities  so  that  a  unique  site  is  not  required  to  control 
smal 1 -sea le  bat t 1 ef ield  training  exercises. 

MODULAR  COMMAND  CENTER  INTERFACE;  Each  trainer  is  connected  to  the 
Battlefield  Command  Center.  This  interface  is  modular  and  interacts 
with  the  trainer  similar  to  the  other  scenario  control  mechanisms 
within  the  trainer. 

TYPES  OF  TRAINERS;  The  types  of  trainers  that  can  be  connected  in  a 
battlefield  training  scenario  include  all  trainer  platforms 
corresponding  to  the  normal  participants  in  a  real  world  battle,  where 
entities  such  as  tanks,  planes,  ships,  and  troops  would  all  be  included 
in  the  exercise.  Tactics  training  is  provided  at  any  level  of  field 
command  and  control.  In  cases  when  a  level  of  command  is  not  taking 
part  in  the  exercise,  the  interactions  with  the  command  level  are 
simulated. 

ARTIFICIAL  INTELLIGENCE:  To  complement  the  AI  requirements  in  Team 
Training,  the  use  of  SAFOR  in  BT  allows  for  the  command,  control,  and 
communications  of  a  large  number  of  platforms  and  their  associated 
weapon  systems.  Useii — friendly  interfaces  are  provided  to  facilitate 
operator’s  control  of  the  SAFOR  system.  As  a  minimum,  SAFOR  monitors 
and  evaluates  all  unit  behaviors,  both  friendly  and  foe,  and  assesses 
the  battlefield  situations  to  react  dynamically  as  events  unfold. 

NETWORKING;  The  BT  environment  consists  of  a  large  number  of 
networked  simulation  devices  (team  trainers,  CBATs,  and  so  forth) 
interacting  within  a  computer-generated  environment.  This 
environment  ranges  from  battle  theatre  down  to  battalion  level.  The 
trainers  interchange  state  information  that  enables  the  individual 
simulators  to  recreate  the  dynamic  effects  of  the  simulated 
environment.  Means  are  provided  to  support  the  updating  of  databases 
for  various  scenarios  on  the  simulation  devices. 

COMMON  SIMULATED  VEHICLES/ENVIRONMENT;  The  vehicles  and 
environment  simulated  in  BT  are  consistent  with  their  simulations  in 
each  of  the  team  trainer  simulators. 

TRAINING  ENVIRONMENT  CONFIGURATION 
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The  r  ec|u  i  r  emen  t  s  above  are  a  basic  set  of  needs  for  an  integr  ated 
training  environment.  In  this  section,  we  identify  existing 
technologies  needed  to  fulfill  the  requirements  for  each  training 
activity.  Special  focus  is  given  to  artificial  intelligence  and 
network i ng . 

CLASSROOM  CONFIGURATION:  The  classroom  configuration  consists  of  a 
group  of  personal  computers  (PCs)  or  workstations  with  their 
associated  peripherals,  connected  in  a  network,  and  a  group  of 
presentation  devices  such  as  an  overhead  projector  ,  video  player,  and 
audio  player  as  part  of  the  IPC.  The  IPC  and  the  student  stations  are 
each  composed  of  a  PC,  a  workstation,  or  a  network  of  PCs  and 
workstations  depending  on  the  processing  requirements  for  the 
application.  Each  station  is  connected  on  a  local  area  network  (LAN) 
to  allow  communications  within  the  classroom  (see  Figure  3).  The 
connection  to  remote  locations  for  transmission  and  receipt  of 
classroom  sessions  can  be  through  wide  area  networks-^  (WANs), 
telephone  lines,  or  satellite  communications  (see  Figure  4). 


The  IPC  contains  a  student  station  integrated  with  media  peripherals, 
so  that  the  instructor  has  all  the  capabilities  of  a  student,  plus  all 
of  the  capabilities  required  to  control  the  class  presentations.  The 
application  code  in  the  IPC  provides  the  instructor  with  a  menu“driven 
control  panel  for  easy  management  of  the  class  flow.  The  IPC 
provides  the  capability  to  develop  a  class  scenario  of  events  that  will 
be  sequenced  through  simple  "next  event"  commands.  The  instructor  is 
provided  with  the  ability  to  interject  the  flow  with  a  different  event 
at  any  time  (that  is,  revisit  material  presented  earlier).  The  events 
in  the  class  scenario  include  commands  to  control  the  various  media 


Figure  3.  Classroom  Configuration 
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Figure  Long  Distance  Training  Configuration 


peripherals  attached  to  the  IPC  control  computer.  The  IPC  activates 
each  device  as  an  event  requires  the  material  contained  on  the 
device.  The  output  is  shown  on  overhead  projection  for  the  entire 
class  to  sees  or  through  presentation  on  each  student’s  display.  The 
ability  to  route  a  student’s  input  to  an  overhead  projection  device  is 
also  under  the  control  of  the  instructor.  Since  a  student  station  is 
part  of  the  IPC  configuration?  the  instructor  can  provide  real-time  or 
canned  exercises  and  explanations  using  tactical  system  simulations. 
The  IPC  can  capture  data  from  the  student  stations  network  to  allow 
real-time  presentation  of  student  responses  to  exercises.  The  IPC 
can  also  connect  to  the  tactical  system  simulator  to  allow  silent 
exercises  (those  that  do  not  affect  the  operations  of  the  simulator 
training)  in  the  classroom  or  debrief  events?  so  that  the  scenario 
data  from  the  simulator  can  be  used  to  drive  the  simulations  in  the 
c 1 assroom . 

The  long  distance  training  configuration  allows  connection  to  another 
classroom  or  just  a  set  of  student  stations  with  a  display  station  for 
viewing  a  classroom  broadcast.  The  ability  to  communicate  from  the 
remote  locations  with  questions  and  exercise  responses  is  essential. 
This  is  accomplished  by  treating  the  remote  locations  as  just  an 
extension  of  the  users  on  the  student  network.  Encoding  and  decoding 
capabilities  are  included  when  security  requires  it. 

STUDENT  STATION  < SS )  CONFIGURATION;  Computer-Based  Autodidactic 
Training  is  accomplished  through  the  student  stations.  The  CBAT 
student  stations  are  used  in  the  classroom  as  described  above  or 
individual  ly  by  students  outside  the  classroom.  The  CBAT  student 
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station  c o nf 1 gur a t i o n  is  a  PC ^  workstation,  or  a  combination  ot  the  two 
connected  together  acting  as  one  device  (see  Figure  5).  The 
processor,  one  or  more  displays,  a  keyboard,  and  a  mouse  are  needed. 
Memo  r  y  and  disk  storage  additions,  printers,  overhead  proj  ec  t i on ,  and 
network  communications  are  included  depending  on  the  specific 
app 1 icat i on . 

The  processing  power  included  in  the  SS  is  dependent  on  the  required 
simulations.  The  intent  of  the  SS  is  to  allow  subsets  of  the  full 
tactical  system  simulator  to  run  on  the  SS.  Uith  this,  hands-on 
training  is  provided  to  students  in  the  classroom  at  the  SS.  Students 
can  reinforce  the  classroom  lectures  with  hands— on  exercises  on  the 
SS  out  of  the  classroom.  The  application  code  in  the  SS  includes 
intelligent  computei — aided  instruction  to  take  the  place  of  the 
instructor.  With  this,  the  student  is  provided  with  the  instructions  to 
conduct  self-paced  study  exercises  and  receive  help  and  feedback  to 
responses.  Various  aspects  of  training  are  included  in  the  CBAT  SS . 
Examples  of  options  to  select  from  are  storyboard  lecture  material, 
canned  exercises  (for  example,  Q&«A  or  simulations),  interactive 
simulations  (for  example,  emulation  of  tactical  system),  and 
prerecorded  lectures  (that  is,  via  multi  — med ia  capability).  For  some 
systems,  the  SS  is  not  able  to  present  the  same  level  of  fidelity  as 
the  tactical  system  simulator  because  that  function  would  require 
expensive  special  purpose  components  in  the  simulator  . 

The  inclusion  of  artificial  intelligence  software  for  intelligent 
computer-aided  instruction,  SAFOR  simulation,  and  natural  language 
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Figure  5.  Student  Station  Configuration 
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processing  are  critical  to  the  usefulness  of  a  SS .  This  issue  will  be 
discussed  later  in  the  AI  section. 

TRSM  TRAINING  SIMULATOR  CONFIGURATIONS  The  configuration  of  each 
simulator  is  dependent  upon  the  tactical  system  being  emulated. 

However?  the  development  of  its  conf i gur a t  i  on  will  have  to  include  the 
needs  of  the  SS  to  facilitate  an  integrated  training  environment. 

This  equates  to  the  development  of  modular  ?  portable  code  to  be  used 
in  the  SS  simulations.  Without  this,  the  cost  of  rewriting  simulation 
code  for  the  SS  will  probably  be  prohibitive.  Also,  interfacing  code  to 
the  classroom  is  part  of  the  simulator’s  design.  The  interfaces  to 
the  classroom  includes  scenario  events  as  described  above  or 
replication  of  student  and  instructor  displays.  The  connections  to 
accomplish  this  are  either  through  direct  connections  from  the 
classroom  to  the  simulator  or  through  a  network. 

Artificial  intelligence  software  for  automated  control  of  forces  plays 
an  important  role  in  presenting  a  common  appearance  to  the  users. 

Common  software  controlling  the  automated  forces  is  reused  from  the 
simulator  to  the  SS  and  classroom  to  guarantee  common  force  behavior 
and  the  ability  to  port  scenarios  between  training  activities. 

BATTLEFIELD  TRAINING  CONFIGURATION;  The  conf i gur a t i on  for 
Battlefield  Training  consists  of  the  tactical  system  simulators  and 
tactics  trainers  playing  an  active  role  in  the  BT  exercise,  a  BT 
Command  Center  (BTCC),  and  any  remote  location  connected  in  a  silent 
role  to  observe  the  exercises  (see  Figure  6).  The  configurations  for 
the  locations  playing  either  an  active  or  silent  role  are  covered 
above  as  simulators  or  student  stations,  with  the  exception  that  the 
hardware  required  to  connect  into  the  BT  network  is  required  along 
with  the  interfacing  software.  The  work  of  the  BTCC  is  accomplished 
using  a  network  of  workstations  with  an  appropriate  number  of 
displays,  networ k/communicat ions  ports,  and  peripherals.  The  BTCC  has 
the  appearance  of  a  mix  between  a  communications  control  center  and 
an  instructor’s  control  console  (as  used  in  a  typical  simulator).  The 
BTCC  operators  use  displays  to  set  up  and  manage  the  participants  of 
the  training  exercise  and  enter  SAFOR  forces  to  simulate  opposing  or 
friendly  forces  not  represented  by  participating  locations.  The 
typical  capabilities  of  an  instructor’s  command  console  and  the 
capabilities  of  the  IPC  are  integrated  in  the  BTCC  to  provide  debrief 
and  instructional  activities  to  participating  locations.  In  this  case, 
the  BTCC  does  not  provide  simulation  replay  for  a  particular  site. 
Instead,  scenario  events  are  provided  as  in  the  BT  exercise, 

ARTIFICIAL  INTELLIGENCE 

The  field  of  artificial  intelligence  (AI)  is  finding  a  considerable 
number  of  applications  in  many  complex  military  problems  as  the 
complexity  of  weapon  systems  increases,  AI ,  in  itself,  is  remarkably 
broad  and  rich  in  the  types  of  techniques  and  methodology  that  it  can 
bring  to  address  the  military  problems  and,  in  particular,  military 
training  problems, C2D  It  is  not  within  the  scope  of  this  paper  to 
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present  a  detailed  description  of  the  technology,  but  rather  to  give  a 
brief  overview  of  the  different  types  of  AI  technologies,  both  proven 
and  maturing  technologies,  and  to  offer  how  these  technologies  may 
solve  the  problems  facing  the  training  community  today.  Specific 
attention  is  paid  to  addressing  the  AI  requirements  as  portrayed  in 
previous  sections. 


EXPERT  SYSTEMS  AND  KNOWLEDGE-BASED  SYSTEMS:  Of  the  various  types 
of  AI  systems,  expert  systems  and  knowledge-based  systems 
technologies  are  widely  perceived  as  the  AI  technologies  with  the  most 
potential  for  the  development  of  near-term  applications.  Much  effort 
has  been  committed  to  use  this  potential  in  emulating  human  expert 
behavior  modeled  after  procedures  and  decision-making  heuristics  as 
employed  by  human  experts.  In  the  following,  we  discuss  a  number  of 
AI  research  areas  revolving  around  expert  systems  or  knowledge-based 
systems  technology: 


Intelligent  Computer-Aided  Instruction 


A  class  of  AI  systems  that  has  recently  received  much 
attention  is  intelligent  computer-aided  instruction.  ICAI  is 
devoted  to  the  development  of  instructional  systems  that  have 
a  true  knowledge  base  of  their  teaching  domain.  These 
systems  offer  the  potential  for  intelligent  systems  to  provide 
adaptive  and  effective  instruction  to  students.  In  a 
classroom  environment,  ICAI  systems  contain  the  knowledge 
required  to  generate  the  correct  answer,  and  therefore  can 
hypothesize  about  what  aspects  of  this  knowledge  the  student 
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is  lacking  so  that  a  strategy  for  instructing  the  student  can 
be  formalized.  Hence,  the  collected  student  responses  to 
eKercises  can  then  be  analyzed  and  fed  back  to  the  students 
during  class  exercises.  ICAI  also  supports  supplemental 
se 1 f - t r a i n i ng  external  to  the  classroom  as  conducted  in 
computer-based  autodidactic  training  by  providing  automated 
instructional  feedback  is  provided  to  the  student. 

2.  Semi —Automated  Force  (SAFOR)  Simulation 

The  application  of  know 1 edge— based  or  expert  systems  can  also 
be  of  great  benefit  in  simulating  intelligent  friendly  and 
opposing  forces  (SAFOR)  to  take  part  in  a  simulated  battlefield 
environment.  SAFOR  simulation  systems  are  equipped  with  an 
appropriate  domain  knowledge  base  that  includes  tactics, 
doctrine,  and  terrain  knowledge  to  dynamically  adapt  or  react 
to  a  simulation  as  new  events  unfold.  SAFOR  controls  target 
movement  and  characteristics  or  behaviors  of  the 
sem i — au toma ted  platforms  in  a  simulated  scenario  presentation 
as  in  the  classroom,  or  CBAT  training  environment.  In  team 
training  or  battlefield  training,  SAFOR  supports  an 
increasingly  larger  number  of  semi— automated  platforms  that 
can  dynamically  respond  and  react  to  a  changing  environment. 

NATURAL  LANGUAGE  PROCESSING?  An  important  area  in  AI  that  can  be 
used  to  address  the  man-machine  interface  is  natural  language 
processing.  Natural  language  processing  refers  to  machine 
understanding  of  the  language  that  people  use  in  a  normal  discourse. 
Two  general  areas  of  research  ares  (1)  understanding  of  written  text, 
and  (2)  understanding  speech. 

Whether  one  is  in  a  classroom,  CBAT  training,  team  training,  or 
battlefield  training  environment,  natural  language  processing 
facilitates  the  interaction  between  user  and  system.  In  a  classroom 
environment,  the  capability  to  carry  an  active  natural  language 
dialogue  expedites  the  process  of  course  delivery  and  allows  the 
student  to  focus  on  the  course  materials  and  not  the  syntax  by  which 
to  communicate  with  the  system.  In  CBAT  training,  this  technology 
greatly  facilitates  the  monitoring  oT  student  performance  and 
instructional  feedback  in  terms  of  natural  language.  In  a  team 
training  or  battlefield  training  environment,  natural  language 
processing  simplifies  the  control  and  communication  processes  of 
SAFOR  simulation  and,  thus,  can  immensely  improve  human  performance. 

AUTOMATED  PLANNING?  Automated  planning  refers  to  the  automated 
ability  to  define  a  sequence  of  actions  (such  as  a  plan)  to  achieve  a 
specific  goal  or  mission.  Much  research  in  AI  has  been  dedicated  to 
the  development  of  systems  for  generating  action  plans  for  various 
kinds  of  agents  faced  with  complex  and  conflicting  goals.  Automated 
planning  forms  a  very  crucial  part  of  SAFOR  simulation  in  that  it 
allows  the  system  to  plan  or  readjust  its  plan  to  achieve  a  mission. 
The  merging  of  automated  planning  technology  and  that  of  game  playing 
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(strategy)  technology  greatly  enriches  the  adversarial  nature  of 
SAFOR  simulation.  This  enhances  the  reactive  power  of  SAFOR  and, 
therefore,  provides  a  more  realistic  opponent  for  all  training 
env i ronments . 

DISTRIBUTED  AI  ;  Distributed  AI  is  the  study  of  how  a  group  of 
individual  intelligent  agents  can  combine  to  solve  a  difficult 
large-scale  problem.  This  concept  is  modeled  after  a  network  of 
experts  required  to  interactively  solve  complex  problem  domains  by 
means  of  coordinating  and  cooperating  within  various  or gan i za t i ons . C 3 3 

A  maturing  area  of  distributed  AI  technology  is  blackboard  technology. 
A  significant  amount  of  work  on  data  fu5ionC4]  has  been  performed 
using  this  technology.  Here,  the  blackboard  serves  as  a  knowledge 
base  wherein  information  collected  from  diverse  sources  are 
interpreted  and  system  behaviors  are  monitored  to  perform  situational 
assessment.  Results  from  this  analysis  can  also  be  used  for 
inferring  likely  consequences  of  a  number  of  given  situations.  This 
is  highly  applicable  in  conducting  classroom  instruction  or 
computer-based  autodidactic  training  where  data  is  gathered  from 
students,  data  analysis  is  achieved,  and  student  performances  are 
monitored  and  evaluated.  Similar  situations  arise  in  a  team  training 
or  battlefield  training  environment  where  battlefield  situations  must 
be  assessed.  This  means  that  all  observable  unit  behaviors  are 
monitored  and  evaluated  to  decide  how  to  conduct  the  battle  forces. 

In  addition,  this  AI  methodology  can  be  used  to  monitor  and  analyze 
student  performances  by  correlating  student  responses  with 
situational  information  residing  on  the  blackboard.  Blackboard 
technology  can  also  be  employed  as  a  decision-aids  tool  for  a  SAFOR 
operator  facing  extremely  complex  situations.  In  such  circumstances, 
this  tool  may  alert  operators  to  possibly  critical  observations  that 
support  the  conduct  of  both  analyses  and  operations. 

NETWORKING 

The  network  requirements  across  the  training  environments  (classroom, 
computer-based  autodidactic,  team,  and  battlefield  training)  have  many 
similarities.  This  section  concentrates  on  the  network  services 
provided  by  the  transport  mechanisms,  such  as  broadcast,  multicast, 
and  transaction.  Even  though  the  user’s  applications  vary  across  the 
training  environments,  there  are  similarities  in  the  application’s 
interface  to  the  transport  mechanisms  that  enables  network 
i nteroperab i 1 i ty  - 151 

BROADCAST  SERVICE:  A  broadcast  service  activates  the  simulation 
exercises  simultaneously  for  the  students  and  receives  responses 
from  the  students’  interaction  with  the  exercises.  The  broadcast 
service  is  a  datagram  or  message  service  that  sends  messages  to  all 
nodes  on  the  physical  network  and  does  not  require  responses  or 
acknowledgements  to  ensure  reliability.  The  redundancy  in  the 
application  combined  with  the  reliability  of  the  physical  netwoi  ks 
ensures  accurate  communications  between  the  simulations.  The 
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broadcast  service  also  supports  the  requirement  for  the  students  to 
exchange  dynamic  changes  in  their  simulations  when  they  interactively 
participate  in  a  simulated  exercise.  The  data  fields  within  the 
broadcast  messages  contain  application  information  that  describes  the 
dynamic  changes  of  the  simulated  environment.  These  data  fields  are 
Distributed  Interactive  Simulation  (DIS)  protocol  data  units  ( PDUs )  that 
are  being  standardized  in  the  Committee  for  the  Interoperability  of 
Defense  Simulations, 

Distributing  the  team  training  functions  for  a  single  platform  has 
similar  communication  requirements  as  the  student  interactions  in 
classroom  simulations.  Although  the  information  within  the  broadcast 
messages  may  be  more  detailed  to  support  the  team  training  functions, 
the  network  broadcast  service  would  be  the  same  in  delivering  the 
messages  to  the  multiple  functions.  The  information  needed  for  the 
distributed  team  training  applications  is  provided  within  the  DIS  PDUs. 
The  broadcast  messages  are  received  by  all  simulation  applications, 
and  the  applications  determine  if  a  particular  message  is  needed  for 
the  function  being  simulated.  The  applications  discard  any  information 
that  is  not  intended  for  its  function,  even  if  it  is  a  complete  PDU . 

Since  there  are  a  limited  number  of  functions  within  a  single  platform, 
this  filtering  of  messages  at  the  application  is  easily  satisfied, 

MULTICAST  SERVICES  Like  the  interactive  student  stations  and  the 
distributed  team  training  functions,  the  d i str ibuted  battlefield 
simulation  also  has  multiple  applications  exchanging  simulation 
information.  The  DIS  PDUs  contain  information  to  dynamically  update 
the  battlefield  simulation  among  multiple  simulated  applications.  In 
this  case,  the  multiple  applications  consist  of  dynamic  entities  in  a 
battlefield  such  as  tanks,  planes,  ships,  and  soldiers.  The  battlefield 
simulation  involves  a  large  number  of  interacting  entities  requiring  a 
large  number  of  messages  to  be  received  and  processed  from  the 
broadcasting  entities.  Having  to  receive  this  large  number  of 
messages  causes  a  processing  bottleneck  for  the  host  simulation. 

Thus,  the  battlefield  training  problem  needs  a  capability  to  filter  the 
broadcast  messages  at  the  network  interface.  A  LAN  addressing 
technique  that  enables  filtering  of  unwanted  broadcast  messages  at 
the  network  interface  is  called  the  multicast  service.  The  multicast 
service  is  a  subset  of  the  broadcast  service  with  the  additional 
capability  of  providing  a  media  access  control  (MAC)  addressing  format 
that  is  used  by  network  interfaces  to  filter  unwanted  messages. 

Using  the  multicast  service,  the  host  only  receives  the  messages  that 
are  destined  for  its  applications,  alleviating  frequent  interrupts  of 
unwanted  incoming  messages.  The  multicast  service  also  enhances 
classroom  training  by  broadcasting  the  student  responses  to  an  IPC 
group  address  only,  so  that  each  student’s  application  is  not  getting 
every  other  students’  responses. 

STREAM  SERVICE;  In  addition  to  the  datagram  services,  the  training 
environments  also  need  connection  or  stream  services.  These 
services  are  used  to  enable  the  exchange  of  large  blocks  of  data  in 
an  ordered  fashion  when  there  are  no  critical  timing  requirements. 

For  example,  the  classroom  environment  needs  to  exchange  databases 
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that  contain  the  course  lecture  between  multiple  widely  distributed 
sites.  Like  the  classroom  environment,  the  team  training  and 
battlefield  environments  also  have  databases  containing  the  static 
information  that  is  used  with  DIS  PDUs  to  create  the  training 
scenarios.  These  databases  can  be  exchanged  before  the  class  or 
training  session  begins  when  there  is  plenty  of  time  to  establish 
connections  and  communicate  the  large  blocks  of  data.  This  service  is 
also  needed  for  instructor  and  student  requests  and  responses  on  an 
individual  basis.  By  having  the  connections  performed  at  initialization 
before  a  class  begins,  this  service  provides  a  reliable  and  efficient 
method  of  individual  interaction  between  the  student  and  the 
i nstruc  tor . 

NETWORK  TECHNOLOGIES:  Future  network  technologies  are  being 
researched  that  will  enhance  the  distributed  training  environments. 
Asynchronous  Transfer  Mode  < ATM )  is  a  future  network  technology  that 
will  significantly  enhance  the  ability  to  satisfy  the  above  service  in 
a  real-time  high  bandwidth  means.  ATM  is  a  packet  switching 
technology  compatible  with  the  data  transmission  technologies  used  in 
today’s  LANs,  and  will  enable  more  efficient  routing  mechanisms 
between  WANs  and  LANs.  By  using  similar  network  services  and 
evolutionary  network  technologies,  the  training  environments  can 
become  integrated  to  provide  more  effective,  seamless  training. 

FUTURE  DIRECTIONS 

We  have  covered  the  integration  of  classroom,  self-paced,  team,  and 
battlefield  training  activities.  An  extension  of  this  integrated 
training  environment  includes  embedded  training  in  the  tactical  system 
and  training  involving  a  combination  of  tactical  systems  with  the  team 
training  simulators.  We  must  address  how  to  integrate  the  events  of  a 
tactical  system  into  a  training  exercise  for  a  simulator  and  how  to 
inject  the  effects  of  the  scenario  from  a  simulator  into  the 
operations  of  a  tactical  system.  A  subset  of  this  concept  involves 
providing  embedded  training  in  the  tactical  system  which  has  common 
links  with  training  provided  by  the  simulator. 

SUMMARY 

Training  improvements  are  realized  with  an  integrated  environment 
that  consists  of  classroom,  autod idac t ic  ,  team,  and  battlefield 
training.  This  integrated  environment  consists  of  existing 
technologies — such  as  AI,  networking,  workstations,  and 
multi-media — and  it  can  grow  as  the  technologies  grow.  The  integrated 
training  environment  relies  on  software  modularity  and  reuse 
practices  to  provide  more  overlap  in  each  phase  to  smooth  the 
transition  between  training  devices  for  personnel.  The  effectiveness 
of  instructors  is  enhanced  by  bringing  hands-on  exercises  into  the 
classroom,  by  extending  instruction  to  include  the  use  of  the  CBAT 
student  stations,  and  by  broadcasting  the  training  lectures  to  other 
locations.  Thus,  the  integrated  training  environment  provides  a  better 
educational  environment  for  the  student  and  allows  training  to  be 
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provided  to  more  students  without  increasing  the  number  of 
i nst rue  tors  o 
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ABSTRACT 

The  requirements  for  a  Computer-Based  Training  ( CBT )  system  have 
evolved  and  were  derived  from  three  major  activities  conducted  by 
I BM  c  They  include  product  development,  industry-wide  CBT 

software  evaluation  studies,  and  program  experience.  The  results 
from  these  activities  directed  the  development  of  the  Generic 
Computer-Based  Training  (GCBT)  system.  The  GCBT  system  was 

developed  by  IBM  Manassas  to  provide  a  multimedia  authoring 
environment  for  interactive  courseware  (ICW)  '  development  and  to 
deliver  the  courseware  in  a  networked  or  stand-alone 

computer-based  training  environment.  By  using  Concurrent 

Object-Oriented  Programming  (COOP)  in  the  GCBT  system,  we 

achieved  modularity  and  code  (courseware)  reusability,  which  are 
inherently  the  byproducts  of  COOP.  The  GCBT  architecture 
provides  a  generic  framework  for  integrating  the  best  of  breed 
Commerc ial-Off-the-Shelf  (COTS)  products  into  the  GCBT  system. 
In  this  paper  5  we  will  discuss  the  architecture  and  the 
operational  capabilities  of  the  GCBT  system. 

INTRODUCTION 

Computer-Based  Training  is  a  powerful  and  effective  tool  as 

demonstrated  in  numerous  applications  throughout  the  training 
industry.  The  Generic  Computer-Based  Training  System  was 
developed  by  IBM  Manassas  to  provide  a  multimedia  authoring 
environment  for  interactive  courseware  development  and  to  deliver 
the  courseware  in  a  networked  or  stand-alone  computer-based 
training  environment.  By  definition,  CBT  supports  the  delivery 
of  curriculum  to  students  in  a  one-on-one  environment.  This 
environment  is  characterized  by  student  sequencing  through  course 
material  in  a  self-paced  manner.  ICW  is  defined  as  training 
material  that  uses  a  computer,  videodisc,  or  other  multi-media 
device,  as  the  primary  basis  of  instruction.  Our  main  goals  in 
developing  the  GCBT  system  are  to: 

Shorten  the  courseware  development  time  through  reusable 
code:  By  employing  the  object-oriented  programming 

methodology,  the  source  code  for  the  CBT  courseware  can  be 
organized  into  a  hierarchy  of  classes  and  subclasses 
(objects).  From  this  library  of  objects,  new  courseware  can 
be  easily  developed  by  reusing  existing  objects.  This  is 
the  inheritance  property  of  object-oriented  programming 
methodology. 
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■  Produce  portable  c  our  se war  e:  GCBT  runs  on  DOS  W i ndo wb  J . o' 

and  OS/2  Presentation  Manager  and  it  employs  ToolBook 

P 

OpenScript  [3]  as  the  authoring  language.  Alsoi  since  IBM 
is  a  member  of  the  Interactive  Multimedia  Association  (IMA), 
the  GCBT  system  has  adopted  the  IMA’s  Practices  for 
Multimedia  Portability  C4]  which  is  also  known  as 
M I L-STD- 1 379D  Appendix  D  (for  portable  courseware). 

■  Effectively  deliver  and  monitor  the  training  objectives:  In 

GCBT,  each  courseware  is  developed  with  its  training 
objectives,  and  its  student  performance  data  are  stored  in 
the  database  which  can  be  used  for  statistical  analysis. 

SYSTEM  ARCHITECTURE 

Through  product  development,  industry-wide  authoring  software 
evaluation  studies,  and  program  experience,  IBM  Manassas 
developed  an  authoring  environment  architecture — the  GCBT 
system — shown  in  Figure  1  that  integrates 

Commerc ial-Dff-the-Shelf  software  and  hardware  products.  This 
architecture  optimizes  the  functionality  of  each  software 
product,  increases  the  user’s  productivity  by  providing  seamless 
integration,  and  provides  modularity  to  allow  the  most 
flexibility  for  growth. 

The  GCBT  system  architecture  and  local  area  network  (LAN) 
topology  is  shown  in  Figure  1  (part  A).  The  GCBT  system  can  be 
configured  as  a  stand-alone  LAN  without  the  Host  connection.  The 
GCBT  architecture  consists  of  El  workstations:  17  Student,  3 

Development,  and  1  File  Server/Print  Spooler.  The  Development 
workstation  provides  the  ICW  development  capabilities,  the 
courseware  is  delivered  at  the  Student  workstation,  and  the  File 
Server/Print  Spooler  provides  file  server  services,  printer 
services  for  the  LAN,  and  acts  as  the  Host  gateway. 

Both  Student  and  Development  workstations  are  configured  with 
similar  hardware  capabilities.  This  provides  development 

functions  for  the  Student  workstations  when  not  in  use  by  the 
students,  and  delivery  functions  for  the  Development  workstations 
when  not  in  use  by  the  ICW  developers.  The  Development 

workstation  represents  an  advanced  version  of  the  Student 
workstation;  however,  the  Student  workstation  does  not  have  the 
video  capture  and  scanning  capabilities. 

The  Student  and  Development  workstations  use  the  same  PS/E  model 
to  assure  the  courseware  developed  and  tested  on  a  Development 
workstation  will  not  incur  performance  degradation  when  run  on 
the  Student  workstation.  The  hardware  configuration  for  the 


^Windows  3.0  is  a  registered  trademark  of  Microsoft  Corp . 
^ToolBook  OpenScript  is  a  registered  trademark  of  Asymetrix 
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workstations  provide  support  for  digitized  audio?  digitized 
video?  full  motion  video?  and  graphics.  The  PS/2  was  selected 
because  of  overall  Micro  Channel  bus  performance?  operating 
system  performance  of  05/2  Extended  Edition?  and  the  exclusive 
video  bus.  The  video  bus  is  important  for  merging  VGA  text  and 
graphics  with  full  motion  video  received  from  the  current 
M-motion  adapter  or  future  DVI^(Digital  Video  Interactive) 
products. 

Full  motion  video  capability  is  provided  using  IBM’s  M-motion 
adapter.  Up  to  three  NTSC  or  two  Super  VHS  video  sources  can  be 
controlled  by  the  adapter.  M-motion  also  allows  for  integration 
of  computer  graphics  with  video/text  overlay  onto  the  video 
images.  The  Video  Capture  adapter  ( VCA )  provides  digitizing? 
editing?  and  storing  of  compressed  video  images.  The  Audio 
Capture  and  Playback  adapter  (ACPA)  provides  high  fidelity  audio 
with  a  sampling  rate  of  44.1  KHz.  The  ACPA  card  can  capture? 
digitize?  store?  and  playback  dynamic  length  audio  segments.  A 
SCSI  adapter  is  used  to  attach  the  Erasable/Rewritable  Optical 
Disk  < EROD )  and  tape  backup  unit.  The  EROD  is  a  means  to  store 
large  amounts  of  data  (for  example?  courseware  material)  on 
removable  media.  All  workstations  will  be  connected  via  a  Token 
Ring  LAN.  The  network  will  be  running  at  16  megabits  per  second. 
The  File  server  will  house  eight  printers  and  contain  the 
database  and  the  libraries. 

SOFTWARE  ARCHITECTURE 

In  this  section?  we  will  first  briefly  introduce  the  concepts  of 
concurrent  object-oriented  programming?  then  present  the  software 
architecture  of  the  GCBT  system.  Finally?  we  will  describe  the 
COOP  implementations  in  the  GCBT  system. 

COOP  CONCEPTS;  Objects  can  be  defined  as  entities  that 
encapsulate  data  and  operations  into  a  single  computational  unit. 
Concurrent  object-oriented  programming  is  a  programming  and 
design  methodology  in  which  the  system  to  be  constructed  is 
modeled  as  a  collection  of  concurrently  executable  modules 
(objects)?  that  interact  with  one  another  by  sending  messages. 
COOP  combines  the  object-based  notion  of  encapsulation?  classes? 
and  inheritance  with  the  concurrent  concepts  of  threads? 
synchronization?  and  communication  Cl?71. 

A  common  semantic  approach  to  modeling  objects  is  to  view  the 
behavior  of  objects  as  functions  of  incoming  communications. 
This  is  the  approach  taken  by  in  the  actor  model  CED .  Actors  are 
self-contained?  interactive?  and  independent  components  of  a 
computing  system  that  communicates  by  asynchronous  message 
passing . 

The  GCBT  system  implements  a  modified  version  of  the  Hewitt-Agha 
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actor-based  model  for  COOP-  By  using  the  COOP  in  the  GCBT 
system?  we  achieved  modularity  and  code  (courseware)  reusability? 
which  are  inherently  the  byproducts  of  COOP. 

GCBT  SOFTWARE  ARCHITECTURE:  The  software  in  the  GCBT  system  is 
composed  of  five  layers  as  shown  in  Figure  1  (part  B).  Layer  1 
contains  the  device  drivers  which  control  all  underlying  hardware 
devices  such  as  the  laserdisc  player  and  the  ACPA  card.  The 
device  drivers  provide  a  set  of  Application  Programming 
Interfaces  (APIs)  that  the  user  can  use  to  request  device 
services  C14I1.  Since  IBM  is  a  member  of  the  IMA?  the  GCBT  system 
has  adopted  the  IMA’s  Recommended  Practices  for  Multimedia 

Portability  (version  1.1)  which  can  be  implemented  in  the  device 
drivers.  Thus?  the  GCBT  system  is  compliant  with  the 

MIL-STD-1379D  Appendix  D  for  portable  courseware. 

Layer  2  is  the  multi-tasking  operating  system  OS/2  CB?11]  which 
provides  system  services  such  as  memory  management? 
process/ thread  management?  and  resource  management. 

Layer  3  adds  more  capabilities  to  the  basic  operating  system 
services  in  QS/B.  Layer  3  has  three  main  components: 

"  Presentation  Manager  (PM)  is  an  object-oriented  tool  C5?103 

that  provides  the  Graphical  User  Interface  (GUI)  with 
windowing  capabilities.  PM  standardizes  a  common  user 
access  ( CUA )  for  all  applications  operating  in  the  OS/2 
environment  C93. 

“  Communication  Manager  C13D  provides  the  connectivity? 
concurrency?  terminal  emulation?  programming  interfaces?  and 
communications  services  for  applications  operating  in  the 
OS/2  environment.  Communication  Manager  supports  the  IBM 
Token  Ring  (IEEE  802.5  &.  802.2)  and  Ethernet  (IEEE  802.3) 

adapters?  from  which  the  software  can  be  written  in  NETBIOS 
or  TCP/IP  protocols. 

•  Database  Manager  C123  is  based  on  the  relational  database 
model.  The  Database  Manager  provides  database  services 
through  the  Structured  Query  Language  (SQL)  which  is  used  to 
define?  query?  and  update  the  database  in  stand-alone  or 
networking  mode. 

Each  component  in  layer  3  provides  the  software  interfaces 
through  a  set  of  standardized  APIs. 

Layer  4  is  the  GCBT  Shell  which  provides  a  generic  form  for 
communication  between  the  applications  in  layer  5  (for  example? 
COTS  products)?  and  for  communication  from  the  non-COTS 
applications  in  layer  5  to  other  layers  (for  example?  Database 
Manager)  in  the  GCBT  system.  The  GCBT  Shell  provides  a 

structured  method  of  cooperation  and  communication  between 
applications  by  employing  the  OS/2  intei  process  communication 
(IPC)  facilities  such  as  semaphores?  signals,  queues?  pipes,  and 
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dynamic  data  exchange  (DDE)  C6,8].  The  GCBT  Shell  standardizes 
the  IPC  process  by  providing  dynamic  link  libraries  (DLLs)  that 
the  COTS  product  can  use  to  communicate  and  coordinate  with  other 
applications.  The  GCBT  Shell  also  provides  a  seamless  user 
interface  across  the  COTS  products  to  make  GCBT  look  and  feel 
like  a  single  product. 

Layer  5  contains  the  COTS  products  such  as; 

ToolBook  C3] 

•  WordPerfect^  ^ 

-  MKS  Revision  ^ontrol  System  (RCS)"^ 

•  Image  Support®  ^ 

•  Audio/Visual  Connection  (AVC) 

•  Corel  Draw® 

"  EasyFlow^ 

Layer  5  also  contains  other  PM  applications,  not  COTS  products, 
such  as  Computer-Managed  Instruction  (CMI),  Video  Server,  and 
Audio  Server. 

ToolBook  is  an  object-oriented  tool.  Each  object  in  ToolBook 
contains  data  and  operations  that  are  written  in  the  ToolBook 
OpenScript  language.  The  user  can  easily  use  ToolBook  to  create 
objects  from  which  ToolBook  can  automatically  generate  the  script 
for  those  objects.  To  ease  the  task  of  ICW  development,  the  GCBT 
system  supplies  a  hierarchy  of  classes  and  subclasses  of  CBT 
objects  such  as  p 1 ayback_f ul 1 _mo t ion_v ideo ,  playback_aud io , 
bookmarking,  pose_quest i on ,  record_answer s ,  and  d i sp 1 ay_gr aph i cs . 
From  this  library  of  objects,  new  courseware  can  be  easily 
developed  by  using  these  existing  objects.  The  reusability  is 
the  inheritance  property  of  object-oriented  programming. 

COOP  IMPLEMENTATIONS;  The  GCBT  system  is  an  actor-based  system. 
For  example,  if  a  courseware  needs  a  full  motion  video  playback 
segment,  the  ICW  author  copies  the  "p  1  ayback_f  u  1 1  _mot  i  on_v  ideo '' 
object  from  the  library  into  his  script.  All  the  author  has  to 
add  are  the  from  and  to  frame  numbers.  At  run  time,  that  object 
will  send  a  message,  which  is  a  Recommended  Practices  [41  command 
such  as  "vdPlay  from=100  to=BOO,"  to  the  Video  Server  (actor) 


^WordPerfect  is  a  registered  trademark  of  WordPerfect  Corp, 
^MKS  RCS  is  a  registered  trademark  of  MKS  Corp. 

^Image  Support  is  a  registered  trademark  of  IBM  Corp. 

^AVC  is  a  registered  trademark  of  IBM  Corp. 

®Corel  Draw  is  a  registered  trademark  of  Corel  Systems  Corp. 
^EasyFlow  is  a  registered  trademark  of  Haven  Tree  Software 
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which  will  playback  the  full  motion  video  segment  accordingly. 
This  modular  design  insulates  the  courseware  from  hardware 
changes  (such  as  the  laserdisc  player)  and  thus  ensures  portable 
courseware.  Furthermore,  since  ToolBook  also  offers  the  runtime 
software  for  DOS/Windows  3.0»  courseware  that  is  developed  on  the 
GCBT  system  will  also  run  on  Windows  without  any  modifications, 
that  is,  it  will  be  portable  across  different  platforms. 

The  GCBT  system  employs  the  MKS  RCS  product  for  revision  control 
of  the  courseware.  However,  the  GCBT  Shell  provides  the 
courseware  configuration  management  ( CCM ) .  The  GCBT  system 
maintains  two  sets  of  tags — Properties  and  Attributes — for  each 
object.  The  Properties  tag  is  created  and  maintained  by  MKS  RCS. 
It  contains  the  change  and  can  include  its  name,  a  text  abstract, 
the  author,  associated  source  code,  object  code,  and  binary 
images.  The  Attributes  tag  is  created  and  maintained  by  the  GCBT 
Shell  and  it  contains  the  EROD  number,  video, disc  number,  course 
number  5  release  number,  operating  system  name,  and  hardware 
platform  name.  At  the  high  level,  each  course  is  an  object  that 
contains  a  group  of  objects  configured  by  the  ICW  author.  In 
this  way,  courseware  configuration  for  new  releases  is  easily 
managed . 

OPERATIONAL  CAPABILITIES 


The  GCBT  system  provides  the  ICW  developer  with  an  assortment  of 
features  to  produce  Level  III  Interactive  lessons  incorporating 
digital  audio  and  video,  and  provisions  for  future  digital  motion 
video  applications.  The  GCBT  system  also  provides  extensive 
capabilities  for  student  tracking 
Interactive  is  defined  as  follows; 
videodisc  applications  must  allow 
computer-generated  text  videodisc 
message— convey i ng  techniques  (such 
computer  text  or  graphics,  or  animated  computer  graphics  mixed 
with  videodisc  images).  Specialized  presentation  components 

(such  as,  digital  audio  and  video)  must  be  fully  integrated  into 
the  presentation  system.  The  GCBT  courseware  can  be  delivered  in 
team— tra i n i ng  mode  or  in  i nd i vi dua 1  — t r a i ni ng  mode  on  either  DS/E 
or  DOS/Windows  platforms. 


and  evaluation.  Level  III 
the  Level  III  interactive 
for  full  integration  of 
information  to  generate 
as,  voice  explanation  over 


GCBT  is  a  database-driven  system.  All  student,  developer,  and 
administrator  information  is  stored  in  the  OS/E  database.  The 
database  includes  information  on  rostering,  courses,  lessons, 
questions,  answers,  objectives,  and  tasks.  DLLs  were  written  to 
provide  access  to  the  database  for  applications  such  as  ToolBook. 
From  the  authoring  environment,  questions  and  answers  are 
downloaded  from  the  database.  When  the  student  exits  the  lesson, 
student  responses  and  other  information  (such  as  bookmark)  is 
stored  in  the  database.  The  GCBT  Shell  invokes  calls  to  the 
database  to  verify  user  access  to  the  different  functions,  list 
courses  and  lessons  for  the  students,  and  maintain  the  database 
for  the  administrator. 
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There  are  three  main  functions  in  the  GCBT  system:  Delivery, 
Development,  and  Administration.  Each  function  can  be  accessed 
via  any  workstation  on  the  LAN  and  is  password  protected  to 
prevent  unauthorized  access.  For  example,  students  do  not  have 
access  to  the  Development  and  Administration  functions.  To  log 
on,  the  user  is  required  to  enter  a  userid  and  password.  The 
user  also  has  the  option  of  changing  the  password  on  this  screen. 
After  the  student  is  successfully  logged  on,  a  course  catalog  is 
displayed.  This  is  a  selectable  list  of  courses  the  student  is 
required  to  complete.  Each  course  has  a  list  of  associated 
lessons  that  is  presented  after  the  student  selects  the  course. 
The  lessons  are  separated  into  three  categories:  Recommended 
Lessons,  In— Progress  Lessons,  and  Completed  Lessons.  Upon 
selecting  a  lesson,  an  information  screen  is  displayed  and  the 
lesson  is  started. 

The  Development  function  provides  tools  to  facilitate  in  the 
development  of  lessons  (courseware).  After  the  developer 
successfully  logs  onto  the  system,  the  development  tools  screen 
is  displayed.  This  is  a  list  of  COTS  products  which  can  be  used 
to  aid  in  the  creation  of  the  lessons.  These  tools  can  be 
replaced  by  any  COTS  product.  The  selectable  tools  are: 
Authoring,  Word  Processing,  Database  Management,  Tape  Backup, 
Flow  Charting,  Graphics,  Revision  Control,  Scanning,  Full  Motion 
Video,  Video  Capture,  and  Audio. 

The  authoring  tool  being  used  is  the  OS/S  version  of  Toolbook. 
DLLs  were  written  to  support  the  integration  of  full  motion 
video,  database  support,  digitized  audio  and  images  into 
Toolbook.  Commands  issued  from  Toolbook  will  invoke  these 
different  functions,  thus  making  the  integration  of  the  lesson 
(courseware)  seamless. 

The  Administration  function  provides  tools  to  facilitate  in  the 
management  of  the  database  and  the  GCBT  system.  After  the 
administrator  successfully  logs  onto  the  system,  the 
administrator  tools  screen  is  displayed.  The  tools  include: 
Word  Processing,  Computer  Management  Instruction  (CMI),  Tape 
Backup,  and  Configuration  Management.  These  tools  can  be 
replaced  with  other  COTS  products. 

The  CMI  provides  standardized  interfaces  to  the  database  so  that 
the  GCBT  system  can  be  tailored  to  specific  training 
requirements.  The  user  is  still  able  to  enter  OS/2  Query  Manager 
and  create  reports,  update  tables,  and  create  queries.  However, 
data  input  and  deletion  can  be  done  using  the  CMI  interface. 
This  includes  functions  to  add  courses,  add  users,  change 
passwords,  delete  questions,  or  list  lessons  as  examples.  These 
functions  were  created  to  also  help  maintain  data  integrity. 

The  CMI  capability  provides  the  user  with  student  administration, 
test,  student  reporting,  and  tracking  of  individual  student 
progress  through  course  of  instruction  (for  example,  test  scores 
and  bookmark).  The  student  administration  involves  student 
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sign~ons  student  lesson  assignment?  and  archiving  student  data. 
Students  are  evaluated  using  tests  consisting  of  true/false, 
multiple  choice?  fill-in?  and  matching  questions.  The  system 
will  record  the  responses  to  all  questions  asked  to  the  students. 
Each  student  will  be  presented  with  a  unique  set  of  questions. 
This  may  be  a  re-sequencing  of  the  questions  or  new  questions. 
Student  reports  provide  information  to  the  instructor  regarding 
the  student’s  progress  and  completion  of  the  training.  These 
reports  are  accessible  by  the  instructor  and  the  student. 

The  GCBT  system  supports  the  Instructional  Systems  Development 
(ISD)  process.  Using  the  Database  Manager,  collection  forms  are 
created  to  aid  in  the  gathering  of  task-specific  information  from 
subject  matter  experts  and  field  observation.  Upon  entering  the 
task  data  into  the  database,  the  course  developer  queries  the 
database  to  analyze  tasks  common  to  more  than  one  job  skill  and 
to  compare  target  population  data  against  task  performance 
requirements.  Based  on  this  task  analysis,  the  course  developer 
decides  which  tasks  are  used  for  training.  The  course  developer 
then  develops  a  set  of  behavioral  objectives  specifying  the 
subtasks,  skills?  and  knowledges  for  each  task  in  the  training 
program.  The  objectives  are  presented  in  a  hierarchical  format 
to  ensure  proper  sequencing  of  training  to  produce  the  most 
learning  in  the  shortest  period  of  time. 

Using  the  products  supplied  in  the  GCBT  system — word  processing? 

flow-charting?  and  database  managei - the  course  developer 

prepares  the  flow  diagrams?  scripts/storyboards?  ICW  evaluation 
items?  production  requirements?  and  the  training  facilitator’s 
guide  for  each  lesson. 

SUMMARY 

The  GCBT  system  is  composed  of  a  user  interface  (the  GCBT  Shell) 
and  COTS  software.  The  user  interface  provides  a  seamless 
environment  containing  the  COTS  products.  This  approach  allows 
for  replacement  of  the  COTS  products  without  any  programming 
effort.  The  COTS  products  were  selected  to  provide  the  different 
aspects  of  authoring  ICW?  such  as,  editor ,  graphics,  authoring 
language?  scanner  support?  flow  chart?  full  motion  video?  and 
digitized  audio.  In  this  way,  the  GCBT  system  can  be  easily 
configured  to  satisfy  current  and  future  customers’  CBT 
requirements.  The  GCBT  courseware  can  be  delivered  in 
t earn— t r a i n i ng  mode  or  in  ind ividual— training  mode  on  either  OS/5 
or  DOS/Windows  platforms. 

By  using  the  COOP  methodology  in  the  GCBT  system?  we  achieved 
modularity  and  code  (courseware)  reusability?  which  are 
inherently  the  byproducts  of  COOP. 

The  GCBT  system  was  selected  by  the  Lockheed  Space  Operations 
Company  ( LSOC ) ?  over  other  commerc i a 1 -of f - the-she 1 f  CBT  systems? 
to  satisfy  their  CBT  requirements.  Currently,  LSOC  is  planning 
to  use  the  GCBT  system?  in  their  mission  with  the  National 
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Aeronautics  and  Space  Administration  to  provide  more  cost 

effective,  timely,  and  efficient  training  tools  for  the  Space 

Shuttle  Launch  Team. 
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Introduction 

This  paper  presents  an  overview  of  visual  simula¬ 
tion  techniques  that  have  been  applied  for  space 
shuttle  part  task  training.  These  techniques  are  also 
directly  applicable  to  the  development  of  part  task 
trainers  for  airborne  weapons  systems.  Specific 
techniques  covered  include  virtual  3D  displays  and 
controls,  camera/sensor  scene  simulation,  and  fimc- 
tional  simulation  of  operatiorul  flight  programs 
(OFPs). 

A  visual  simulation  based  part  task  trainer  was 
recently  developed  for  the  NASA  Johnson  Space 
Center  imder  a  maimed  spaceflight  simulation  re¬ 
search  program.  The  Rmidezvous  Radar  Trainer  is 
a  workstation  based  part  task  trainer  that  is  designed 
for  instructing  pilot-pool  level  astronauts  in  the 
operation  of  the  Ku-band  rendezvous  radar  system  on 
board  the  space  shuttle.  The  rendezvous  radar 
system  features  both  manual  and  operational  flight 
program  control  of  the  radar  mechanical  assemblies 
and  electronics,  man-in-the-loop  control  through 
hand-eye  coordination  and  tracking,  multiple  concur¬ 
rent  display  output  devices,  and  crew-flight  program 
interfacing  through  keypads  and  CRT’s. 

This  paper  provides  a  detailed  discussion  of  the 
trainers  design,  including  how  visual  simulation  was 
used  to  satisfy  part  task  training  objectives.  The 
trainer  provides  fuU  control  over  simulated  payload 
bay  cameras,  allowing  the  student  to  visually  inspect 
the  radar  assembly,  the  payload  bay  doors,  and 
individual  payloads.  This  visual  capability  aids  in 
student  understanding  of  system  operations.  In 
addition,  students  are  able  to  correlate  instrument 
readings  with  physical  equipment  conditions  as  seen 
through  the  camera.  Development  cost  for  the 
rendezvous  radar  trainer  was  minimized  through  the 
use  of  a  fimctional  simulation  approach  that  focused 
on  tailoring  the  software  architecture  to  support 
specific  training  objectives.  Suggestions  for  extend¬ 
ing  the  visual  part  task  trainer  approach  to  cover 
airborne  weapon  system  applications  in  the  areas  of 
sensor  simulation,  weather  visualization,  and  terrain 
rendering  are  then  presented  for  discussion. 


Visual  Simulation  Alternatives  for  Part  Task  Training 

As  full  up  operational  flight  trainers  (OFTs)  have 
become  more  complex  to  accommodate  mission 
rehearsal  demands,  many  end  users  have  become  in¬ 
creasingly  more  interested  in  low  cost  man-in-the- 
loop  simulation  alternatives  for  part  task  training. 
Part  task  trainers  can  provide  a  cost  effective  alterna¬ 
tive  to  OFTs  for  providing  crew  members  with 
hands-on  training  in  a  variety  of  mission  critical,  time 
dependent  tasks.  Currently,  four  methodologies  are 
available  for  implemoiting  visual  simulation  functions 
within  part  task  trainers.  These  are:  1)  hardware 
stimulation,  2)  hardware  emulation,  3)  architecture 
emulation,  and  4)  functional  simulation. 

Hardware  Stimulation 

Under  the  hardware  stimulation  approach,  visual 
simulation  functions  are  provided  through  the  use  of 
operational  mission  equipment  within  the  part  task 
trainer  design.  An  example  would  be  the  incorpora¬ 
tion  of  an  aircraft  digital  image  tracker  ubereby  the 
tracker  is  stimulated  with  computer  generated  imag¬ 
ery  and  control  signals  from  the  PTT  host  computer. 
This  approach  has  the  advantage  of  increased  validity 
in  terms  of  trainer  performance,  but  has  disadvantag¬ 
es  in  the  areas  of  development  cost,  and  reliabili¬ 
ty/maintainability. 

Hardware  Emulation 

V^th  the  hardware  emulation  approach,  the  visual 
simulation  functions  are  emulated  within  the  PTT 
through  the  use  of  commercial  hardware  that  is 
functionally  equivalent  to  the  mission  equipment.  An 
example  of  this  approach  would  be  the  use  of  a  com¬ 
mercial-off-the-shelf  image  processor  in  place  of  the 
aircraft  digital  image  tracker  within  the  PTT.  This 
method  results  in  improved  reliability,  but  still  results 
in  a  complex,  expensive  design  approach.  In  addi¬ 
tion,  maintaining  concurrency  with  the  aircraft  may 
be  difficult  due  to  either  peculiarities  of  the  image 
processor  or  unique  techniques  that  may  be  required 
to  implement  aircraft  changes. 
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Architecture  Emulation 

An  approach  aimed  towards  improving  concur¬ 
rency  between  the  PTT  and  the  operational  equipment 
is  to  emulate  the  architecture  of  the  operational 
equipment  within  the  PTT  design.  Under  this  ap¬ 
proach,  each  component  of  the  applicable  mission 
equipment  is  emulated  within  the  PTT  in  terms  of 
function  and  interfaces.  This  method,  typically  used 
for  flight  simulator  design,  inqiroves  on  the  ability  to 
track  configuration  changes,  but  results  in  a  cumber¬ 
some  PTT  design  that  emphasizes  configuration 
validity  over  training  effectiveness. 

Functional  Simulation 

A  fourth  iqjproach  is  to  deemphasize  the  opera¬ 
tional  equipment  configuration  and  simulate  only 
those  functions  that  are  required  to  support  part  task 
training  objectives.  This  is  the  approach  that  was 
taken  for  the  NASA  rendezvous  radar  trainer.  Since 
the  PTT  design  is  based  on  functional  performance, 
the  PTT  is  only  impacted  by  those  operational  equip¬ 
ment  changes  that  result  in  performance  differences 
that  are  visible  to  the  crew. 

A  software  intensive  approach,  functional  simula¬ 
tion  provides  a  means  for  designing  PTT’s  that  are 
low  cost,  focus  on  end  user  requirements,  are  easy  to 
update,  and  may  be  tailored  and  reused  for  similar 
type  mission  equipment  training. 

With  the  functional  simulation  approach,  PTT 
training  objectives  are  defined  in  light  of  the  opera¬ 
tional  system  capabilities  for  the  mission  equipment, 
and  PTT  software  requirements  are  then  derived  from 
the  training  objectives.  This  structure  provides  for 
requirements  traceability  and  provides  a  direct  means 
for  correlating  development  cost  with  traiuing  objec¬ 
tive. 


Functional  Simulation  Atmlied 

The  SwRI  developed  rendezvous  radar  trainer 
was  designed  for  NASA  using  a  number  of  visual 
simulation  techniques  that  are  extensible  to  airborne 
weapons  training.  The  follcwing  presents  an  over¬ 
view  of  the  functional  simulation  approach  as  applied 
to  this  trainer. 


Operational  System  Description 

The  rendezvous  radar  trainer  supfmrts  mission 
critical  training  objectives  in  the  operation  of  the  Ku 
band  system  on  board  the  space  shuttle.  Figures  A 
and  B  show  the  location  and  configuration  of  the  Ku 
band  antenna  on  board  the  shuttle.  The  Ku  band 
system  serves  as  both  a  rendezvous  radar  and  as  a 
communication  link  to  the  TDRS  satellites.  Crew 
members  operate  the  rendezvous  radar  system  from 
the  aft  flight  deck  of  the  shuttle  as  shown  in  Figure 

C.  The  rendezvous  radar  system  features  character¬ 
istics  that  are  similar  to  many  airborne  weapons 
systems  including  operational  flight  program  control, 
man-in-the-loop  control  and  multiple  interactive 
displays  and  controls. 

Part  Task  Training  Objectives 

An  example  of  training  objective  decomposition 
into  tasks  for  the  rendezvous  radar  is  given  in  Figure 

D.  For  the  objective  ’Operate  Rendezvous  Radar*, 
five  tasks  have  been  identified,  each  task  defined  by 
a  set  of  conditions,  procedures,  and  standards  for  task 
performance.  The  conditions  define  the  training 
situation  in  terms  of  displays  and  controls.  Proce¬ 
dures  describe  how  the  student  is  expected  to  interact 
with  the  conditions,  and  the  standard  defines  the 
criteria  for  successful  task  accomplishment. 

Derived  PTT  Requirements 

Once  all  training  objectives  and  subsequent  tasks 
were  identified  for  the  rendezvous  radar,  specific 
trainer  requirements  to  support  the  training  objectives 
were  defined.  It  was  mutually  agreed  upon  that  each 
task  would  be  implemented  as  a  selectable  trainer 
lesson.  When  the  student  selected  a  lesson,  they 
would  be  presented  with  all  displays  and  controls 
necessary  to  perform  the  task  (conditions),  and  their 
procedures  would  be  automatically  tracked  to  assess 
task  performance  (standard).  In  addition,  initial 
condition  sets  were  determined,  instructor  selectable 
malfunctions  were  defined,  and  trainer  controls  were 
selected. 

Functional  Design 

The  trainer  was  designed  as  a  visual  simulation 
application  for  execution  on  any  Silicon  Graphics 
woricstation.  It  was  desired  that  special  hardware  or 
software  not  be  required  to  host  the  rendezvous  radar 
trainer.  This  is  one  of  the  key  strengths  that  visual 
simulation  technology  has  to  offer  for  part  task 
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tiaimer  deveiopmeat.  In  additiom,  it  was  destired  that 
no  programming  knowledge  or  special  mstractioias  be 
require  ora  the  part  of  the  student  to  operate  the 
trainer. 

Software  r^uiretraerats  were  grouped  into  two 
distinct  as^s;  1)  user  interface  and  2)  system  simaSa- 
tioffi.  This  separatioH  was  designed  to  aUow  maxi¬ 
mum  flexibility  in  tailoring  the  trainer  to  meet  future 
nmis,  such  as  incorporating  actual  equipment  dis¬ 
plays  or  additional  simulation  models.  Through  this 
approach^  visual  simulatiora  technique  via  virtual 
displays  and  controls  cam  be  effectively  implemsrated 
to  prototype  or  pm&ie  the  use  of  mission  equipm^t. 
Figure  E  illustrate  the  high  level  design  of  the 
rendezvous  radar  trainerj  including  allocation  of  the 
major  software  components. 

Simulation  Modeling  Approach 

The  rendezvous  radar  trainer  software  include 
four  real-time  simulation  models:  1)  Radar  Models  2) 
On  Board  Computer  Model,  3)  Rendezvous  Model, 
and  4)  CCTV  Model.  These  models  are  time  sliced 
and  updated  at  a  IS  Hz  rate  in  response  to  stud^t 
interactions.  An  ex®:utive  program  controls  model 
execution  and  directs  model  to  model  commumrataon 
through  shared  memory. 

The  Radar  Model  simulates  the  operating  charac¬ 
teristics  of  the  Ku  band  system,  including  mod® 
control,  signal  procming,  and  antenna  n^chani^ 
assembly  dynamics.  The  On  Board  Conqjuter  Model 
simulates  the  operational  flight  program  controOed 
functions  pertaining  to  the  radar,  including  keypad 
syntax,  mode  control,  srassor  integration,  and  mal¬ 
function  logic.  The  Rendezvous  Model  simulates  the 
flight  traj^tory  of  the  shuttle  and  a  rraidezvous  target 
in  a  typical  shuttle  orbit  through  a  simple  three  body 
orbital  mechanics  algorithm.  The  CCTV  Model 
renders  visual  for  any  of  the  cameiM  located 
in  the  payload  bay.  Visual  f^tus^  otodeled  include 
the  Ku  band  antenna  n^hanical  assembli^,  payload 
bay  interior,  and  the  payload  bay  djHjrs. 

Trainer  Metrics 

The  rendezvous  radar  trainer  was  designed, 
implemented,  and  validated  within  a  period  of  nine 
months  by  a  tfiam  of  thr^  simulation  engineers.  The 
baseline  trainer  software  configuration  consists  of 
20,000  lines  of  new  development  source  code  written 
in  the  C  language. 


Extension  to  Airborne  Weapons  Training 

Many  of  the  visual  simulation  concepts  utilized 
for  the  rendezvous  radar  trainer  are  high  extensible  to 
airborne  weapons  training,  the  virtual  displays  and 
controls,  cooperating  simulatioB  models,  and  visual 
scene  simulation  technique  utilize  for  die  rendez¬ 
vous  radar  trainer  may  all  be  tailored  and  reused  for 
application  to  airborne  w^pons  training.  Improved 
workstation  bas^  Ada  development  tools  along  with 
emerging  standards  in  the  ar^  of  graphics  libraries 
(GL,  PEX)  and  POSIX  compliant  application  pro¬ 
gram  interfaces  (API’s)  may  now  allw  visual  simula¬ 
tion  models  to  be  developed  that  are  transportable  and 
reusable  across  a  wide  range  of  workstation  platforms 
and  end  user  application  areas. 

Sensor  Simulation 

With  the  aide  of  these  emerging  standards,  the 
functional  simulation  approach  may  1^  appli^  to 
develop  training  effective  sensor  simulation  models 
for  satisfying  part  task  training  objectives  across  a 
number  of  werqions  systems.  Visual  simulation  based 
software  models  developed  via  functional  simulation 
offer  many  benefits  in  that  development  cost  is  low, 
they  may  be  readily  reus^,  and  they  only  need  to 
TTiaintaiin  concurrency  with  of^rational  equipment 
performance,  as  oppored  to  ^uipment  configuration. 
Example  of  visual  simulation  based  sensor  models  in 
support  of  man-in-the-loop  part  task  training  include 
radar  warning  receivers,  multimode  fire  control 
radars,  FUR  sensors,  missile  approach  warning 
systems,  and  weapons  seekers. 

Tactics  Analysis 

Workstation  based  visual  simulation  applications 
may  also  be  effiretively  used  to  analyze  and  evaluate 
tactical  scenarios  for  both  mission  planning  and  post¬ 
action  debriefing.  SwRI  is  currently  developing  a 
woricstation  based  tactics  analysis  tool  for  an  Air 
Force  sjronsor  that  allows  mission  analysts  to  interac¬ 
tively  view  multi-aircraft  scenarios. 

Weather  Visualization 

Emerging  visual  simulation  technique  in  the 
areas  of  volume  visualization  and  increasing  work¬ 
station  performance  may  now  allow  3D  weather 
effects  to  be  simulated  for  part  task  training.  One 
promising  approach  is  to  render  visual  wither  scenes 
based  on  doppler  radar  or  super  computer  based  data 
Q  9  sets  that  describe  weather  conditions  within  a  specific 


volume  of  airspace  over  a  given  period  of  time. 
SwRI  is  pursuing  this  approach  for  the  visualization 
of  noicroburst  windshear  conditions,  but  the  technique 
may  also  be  applicable  for  incorporating  combat 
weather  conditions  into  man-in-the-loop  trainers. 

Terrain  Rendering 

Workstation  based  part  task  trainers  utilizing 
visual  simulation  technology  are  also  well  suited  for 
accommodating  emerging  digital  terrain  data  sources 
for  visualizing  ground  environments.  Powerful 
graphics  architectures  and  multi-processor  capabilities 
allow  workstations  to  produce  highly  detailed  terrain 
imagery  based  on  emerging  types  of  digital  source 
data  including  Multispectral  Imagery  (MSI)  and 
Tactical  Terrain  Data  (TTD). 

Summary 

Visual  simulation  technology  has  advanced  to  the 
point  where  many  training  features  that  have  tradi¬ 
tionally  been  reserved  for  operational  flight  trainers 
are  now  available  for  incorporation  in  part  task 
trainers.  Virtual  displays  and  controls,  out  the 
window  scene  generation,  sensor  simulation,  and  on 
board  computer  modeling  may  all  be  applied  within 
man-in-the-loop  oriented  part  task  trainers.  Through 
the  use  of  the  functional  simulation  development 
methodology  as  presented  here,  these  features  can  be 
developed  ,  to  maximize  training  effectiveness  while 
minimizing  trainer  development  cost.  In  addition, 
workstation  ba^  visual  simulation  capabilities  are 
nq>idly  emerging  that  allow  new  training  features 
such  as  weather  simulation  and  new  digital  terrain 
data  products  to  be  acconunodated  for  part  task 
training. 


110 


Panel  A7 
CCTV  controls 


1)  OmanutH  OK  moat 
opottoon 


2)  Potierm  Rodai  ntvrgtiien 
pneoduto 


OBJECTIVE  <2:  0;^rate  Rendezvous  Radar 


PRXQDURES 


OamonevaM  tulum  ef  QPC  mgb* 
nctudmg  wio  tMicX  and  vacti. 
coast  itirougn  otiaeuraiion  sons, 
auto  raacdura  atiar  broak,  and 
angla  wandsr  adisn  tiM  >  30  dag. 


Track  largat  and  updaia  nangalion 
Hlar  «nth  radar  data.  Monitor  and 
aaaaaa  ttOtiinm  data  on  CRT 


3 )  Oainai—  GPC  OESKS  mods 
Oparadon 


4 )  Domomnm  AUTOMVi  modo 
oporakons 


5)  OameiMPaW  Protimilr 
Oponlem  Radtr  Conm 


CONOmONS 

II 

CRT 

(OPS  201) 

(rmillipta 

•c*Aanof) 

Z  mndvm 
iarg«l  viewing 

Pansl  A1U 
-Pml  A3 

CRT 

(SPEC  33) 

•  Z  wmoew 
targai  viewing 

-Pmal  A1U 
•Pml  A2 

(mulllpl* 

mntnM) 

CRT 

(OPS  201) 

■2  window 
target  vieeeng 

-Pwial  AlU 
•Pml  A2 

-CRT 
(OPS  201) 

(muHIpla 

-  2  wtndew 
target  vmng 

-Pml  A3 
-Panal  A1U 

-CRT 

(OPS  301) 

CCTVOA 

ananabon 

Oantenadata  GPC  OCSU  loaMsa 
including  2  aaeond  angla  updaia. 
ranga  aaaren.  and  oanuol  loaa  mlian 
poHidng  lirough  IM  sOacuiaSen 


Oamcnaaala  AUTO  and  MAN  mods 
laaluias  including  mtinmti*  scan 
aaaicft,  dual  alsw  ralaa,  manual 


mdudbig  scan  isam  legle.  gkrPal 
Mia.  and  RF  poM  managsmanl 


STANDARD 


AS  GPC  mods 
fsaturss  oicarsisad 


Good  radar  data 
succaaahdlr 
translsrrad  to  NAV 
Propogalad  Oau 
program 


Al  AUTOTMAN 
cliaracteriaiics 
damonatratod 


Sals  uaagsol  dis 
radar  in  proiirntty 


FiguK  D  •  Example  Traimng  Objective  DecompositioB 


Hie  Threat  Matrix: 
AHypermedia  Lesson  That  Teaches  Facts 
About  the  Naval  Faroes  of  the  World 


Michael  R.  Simonson 
Professor 

and 

Robert  Kelly 
Graduate  Student 

Curriculum  and  Instructional  Technology 
College  of  Education 
Iowa  State  University 
Ames,  Iowa  50011 
(515)204-6840 


113 


The  TTireat  Matrix: 

A  Hypermedia  Lesson  That  Teaches  Facts 
About  the  Naval  Forces  of  the  World 


In  1988,  Iowa  State  University's  College  of  Education  and  Media  Resources  Center  (Ames,  Iowa), 
contracted  with  Oak  Ridge  Associated  Universities,  Inc.  (Oak  Ridge,  Tennessee)  to  develop  a  plan 
to  promote  the  increased  use  of  educational  technology  by  the  instructors  and  students  of  the  United 
States  Navy's  Surface  Warfare  Officers  School  (SWOS).  Located  in  Newport,  Rhode  Island,  the 
mission  of  SWOS  is  to: 

•  provide  the  Naval  Surface  Warfare  Force,  through  a  system  of  functional  training,  with 
officers  professionally  qualified  to  serve  as  effective  naval  leaders  on  surface  warfare 
ships  with  the  ultimate  goal  of  Command-at-Sea. 

•  serve  as  a  focal  point  for  development  and  integration  of  qualification  standards  and 
functional  training  in  support  of  the  established  continuum  of  Surface  Warfare  Officer 
professional  and  billet  specialty  training. 

One  of  the  curriculum  areas  at  SWOS  is  designed  to  prepare  officers  to  work  in  a  ship’s  "Combat 
Information  Center"  (CIC),  where  they  have  the  responsibility  of  interpreting  information 
collected  from  electronic  sensors  and  human  look-outs.  When  a  "contact”  is  made  by  the  ship's 
radar,  or  from  a  surveillance  plane,  it  is  the  responsibility  of  this  officer  to  determine  what  the 
"contact"  is,  and  to  determine  its  threat. 

One  of  the  basic  skills  needed  by  a  CIC  officer  is  the  ability  to  recall  the  names  and  characteristics 
of  the  warships  and  aircraft  of  the  fleets  of  the  nations  of  the  world.  Initially,  this  information  is 
memorized,  and  SWOS  students  are  tested  on  their  ability  to  remember  this  data.  Later,  students 
apply  their  factual  knowledge  to  realistic  simulations  of  threat  situations. 

To  facilitate  learning,  SWOS  instructors  and  students  organize  the  "threat"  information  into  a 
chart  with  columns  (ships)  and  rows  (characteristics)  and  they  refer  to  it  as  the  "threat-matrix" 
(TM).  The  TM,  as  it  is  presented  by  SWOS  instructors,  contains  thousands  of  facts  that  students 
must  memorize,  including: 

•  colored  pictures,  usually  slides,  of  ships,  aircraft  and  weapons  (e.g.  missiles), 

•  line  drawings  of  ships,  aircraft  and  weapons. 
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®  names  of  ships,  aircraft  and  weapons, 

®  characteristics  of  ships,  aircraft  and  weapons  (e.g.  speed,  weight,  electronic 
characteristics), 

®  relationships  between  other  categories  of  information. 

Once  the  data  in  the  TM  are  memorised,  students  apply  their  knowledge  in  instructional 
simulations  that  evaluate  their  ability  to  function  in  a  ship's  CIC. 

One  important  characteristic  of  the  TM  is  the  considerable  overlap  of  the  information  contained  in 
it.  Ships  and  aircraft  share  weapons  systems  and  sensors,  and  weapons  carried  by  aircraft  and 
ships  have  characteristics  such  as  speed  and  range,  as  do  the  platforms  themselves.  Part  of  the 
difficulty  involved  in  memorizing  the  facts  of  the  TM  is  relating,  or  linking,  facts  to  the  various 
platforms  where  they  are  found.  Officers  traditionally  memorize  the  information  of  the  TM  using 
flash  cards,  peer  teaching  groups,  individual  memorization,  and  other  non-technological  means. 


The  Hypermedia  Lesson 

Hypermedia  is  a  relatively  new  term  that  is  often  defined  as  the  use  of  computer-based  systems  to 
present  a  variety  of  media  in  a  non-linear,  randomly  accessible  manner.  Hypermedia  is  often 
confused  with  multimedia  which  is  similar  but  considered  by  most  to  be  a  subcategory  of 
hypermedia.  The  linking  capabilities  of  hypermedia  environments  permit  learners  to  control 
their  accessing  of  information,  and  to  leam  in  a  manner  that  is  individual  and  unique  for  them, 
rather  than  in  a  predetermined,  instructor  planned  way.  There  is  even  some  research  to  support 
the  contention  that  within  certain  parameters  learners  will  select  the  best  technique  for  their 
personal  styles  of  learning.  Since  hypermedia  systems  with  their  linking  and  branching 
capabilities  mirror  the  organization  of  th®  content  of  the  threat  matrix,  and  since  naval  officers 
leam  the  TM  using  a  variety  of  techniques,  it  was  decided  that  a  hypermedia  lesson  would  make 
an  excellent  tool  for  teaching  and  learning  about  the  TM,  and  would  ultimately  become  a  rich 
environment  in  which  to  study  how  naval  officers  leam  the  TM. 

In  1991,  a  prototype  hypermedia  lesson  called  "The  Threat  Matrix:  A  Computer  Based  Lesson  (TM 
LESSON)"  was  developed.  Later  in  1991,  the  lesson's  name  was  changed  to  the  "Shipboard 
Systems  Matrix:  A  Computer  Based  Lesson.  The  prototype  lesson  had  the  following 
characteristics: 


115 


1.  It  was  constructed  as  a  "shell"  lesson  that  could  be  easily  updated  and  changed.  Specifically, 
the  outline  of  the  lesson  was  constructed  by  ISU  instructional  developers.  The  factual  information 
inserted  into  this  outline  was  obtained  from  the  unclassified  book,  Combat  Fleets  of  the  World. 

The  lesson  was  constructed  so  that  SWOS  officers  could  easily  replace  unclassified  data,  such  as 
the  speed  of  the  Soviet  warship  KIEV,  with  classified  information.  IBM's  recently  introduced 
software  package  LINKWAY  was  selected  as  the  authoring  system  for  the  lesson.  LINKWAY  is 
similar  to  Apple's  HyperCard.  Both  software  packages  are  inexpensive,  require  no  modification  to 
hardware,  are  widely  available,  and  are  considered  excellent  examples  of  hypermedia  authoring 
systems. 

2.  It  was  MS-DOS  based.  Since  the  designated  computer  of  the  US  Navy  at  that  time  was  the 
Zenith  model  248,  it  was  the  computer  used  to  design  the  lesson.  The  lesson,  with  few  exceptions, 
was  developed  to  be  "transportable"  to  any  MS-DOS  computer,  including  computers  using  80386 
central  processors. 

3.  It  was  designed  to  be  intuitive  and  easy  to  use.  The  metaphor  of  a  "book"  was  identified  as  the 
theme  for  the  organization  of  the  lesson.  In  other  words,  the  lesson  had  a  cover,  a  preface,  a  table  of 
contents,  chapters,  self  testing  sections,  and  an  index.  The  book  analogy  was  selected  because 
Combat  Fleets  of  the  World  was  a  reference  Navy  officers  were  familiar  with  and  used  to  help 
them  learn  about  the  TM.  This  made  the  organization  of  the  lesson  straightforward  which  made  it 
more  likely  that  students  would  concentrate  on  learning  the  TM,  rather  than  on  how  to  use  the 
computer  lesson.  Additionally,  the  lesson  was  graphically  oriented  and  "mouse-driven."  All 
commands  were  activated  by  pointing  the  mouse  at  a  portion  of  the  screen  and  clicking  one  button. 
The  computer  keyboard  was  almost  never  needed.  Finally,  a  standard  screen  layout  was  selected 
for  all  of  the  lesson's  pages.  Identification  information  was  always  placed  in  the  upper  left  comer 
of  the  screen,  navigation  information  for  movement  through  the  lesson  was  always  placed  in  the 
upper  right  portion  of  the  screen,  and  TM  visuals  and  data  were  always  placed  in  the  center  and 
bottom  two  thirds  of  each  screen.  In  other  words,  the  lesson  was  designed  so  students  would  quickly 
become  familiar  with  its  organization.  The  intent  was  to  make  the  delivery  system  "transparent," 
so  attention  could  be  placed  on  the  information  of  the  threat  matrix. 

4.  It  included  various  types  of  visual  information  which  were  used  as  mnemonics  to  help  students 
remember  facts.  Line  drawings  of  ships,  aircraft,  weapons,  and  sensors  were  included  and  were 
presented  on  the  computer's  screen.  Additionally,  the  system  included  a  videodisk  player 
connected  to  and  controlled  by  the  computer.  It  played  a  videodisk  containing  thousands  of  still 
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Formative  Evaluation 

The  prototj^e  lesson  was  completed  in  early  1991,  and  delivered  ready  to  use  to  SWOS  by  ISU 
instructional  developers.  Immediately,  a  SWOS  instructor  updated  the  lesson  with  the  appropriate 
classified  facts  of  the  TM.  The  instructor  was  given  approximately  two  hours  of  in-service  in  the 
operation  of  LINKWAY  and  the  use  of  the  lessom  Because  of  the  way  in  which  the  lesson  was 
created  it  was  relatively  simple  for  the  SWOS  instructor  to  modily  with  classified  data  the 
unclassified  version  of  the  lesson  created  by  ISU  instructional  developers. 

Next,  the  lesson  was  made  available  to  SWOS  students  enrolled  in  the  Department  Head  School's 
CIC  curriculum.  They  were  not  required  to  use  the  lesson.  Rather,  it  was  an  option  for  learning  the 
TM  that  was  made  available  to  them.  During  the  period  of  the  course  and  following  its  completion, 
evaluation  data  were  collected.  In  all  cases  the  data  were  positive.  Two  comments  were  prevalent. 
First,  students  wanted  a  complete  version  of  the  lesson,  and  second,  students  asked  for  more  copies 
of  the  system  so  several  of  them  could  learn  simultaneously.  Additionally,  input  from  SWOS 
instructors  were  solicited.  Their  comments  concentrated  almost  exclusively  on  the  content  of  the 
lesson,  not  on  its  organization  or  delivery. 


Because  of  the  positive  reactions  of  students  and  instructors,  it  was  decided  that  the  lesson  should  be 
completed  and  formally  evaluated.  Currently,  ISU  instructional  developers  are  working  closely 
with  SWOS  instructors  to  update  and  complete  the  lesson,  now  called  the  "Shipboard  Systems 
Matrix  (SSM)."  The  SSM  will  be  completed  by  early  summer,  1992. 
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Enclosures: 


#1  The  Threat  Matrix  Book 

#2  Section:  USSR 

#3  Chapter  1:  Surface  Warships 
#4  Chapter  5:  Sensors 
#5  Chapter  6:  Weapons  Systems 
#6  Sample  Quiz  Item:  Drawings 
#7  Sample  Quiz  Item:  Video  Recognition 
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Chapters  Completed 


This  section  shows  exactly  where 
users  are  in  the  lessson.  It  also  shows 
the  user  how  this  lesson  example  fits 
into  the  total  "Threat  Matrix." 


Clicking  on  the  first  five  buttons  will 
take  you  to  that  identified  section. 
Clicking  on  "PREVIOUS  SCREEN"  will 
allow  the  user  to  retrace  you  moves,  and 
see  up  to  12  previous  screens. 


Chapter  5,  'Sensorsf ' 
Example  page  from  lesson 


Cognitive  Map 

This  section  shows  exactly  where 
users  are  in  the  lessson.  It  also  shows 
the  user  how  this  lesson  example  fits 
into  the  total  "Threat  Matrix." 
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Navigation  Buttons 
Clicking  on  the  first  five  buttons  will 
take  you  to  that  identified  section. 
Clicking  on  "PREVIOUS  SCREEN"  will 
allow  the  user  to  retrace  you  moves,  and 
see  up  to  12  previous  screens. 


Line  Drawing 

Salable  for  this 
pie,  a  scanned 
Irawing  will  be 
shown. 


SECT  ion:  USSR 

CHAPTER  5:  Sensors 

LESSON  5A:  Long  Range  Air  Searcli 

EXANPLE  »Hll:  Top  Sail 


®  4 


COVER 
PREFACE 
BOOR  CONTENTS 

CHAPTER  comsms 

INREX 

PREVIOUS  SCREDT 


Mntn 
•mt  mi 


Fa»<  Or 

IMRiticmai 

InfonutlM 

Skip  Classes 

Information  Pop-up  Buttons 
Click  on  either  of  these  buttons  to  read  more  about  its 
contents.  When  finished  reading,  click  on  the  upper 
left  corner  of  the  pop-up  box,  and  it  will  go  away. 


Video  Buttons 
If  a  green  video  icon 
is  shown,  clicking  on 
it  will  show  a  motion 
video  sequence  from 
the  videodisc.  A  red 
icon  will  show  a  still 
frame. 

"TURN  OFF  VIDEO 
STILL"  gives  the  user 
control  of  how  long  the 
video  still  image  is  on 
the  screen 
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If  a  green  video  icon 
is  shown,  clicking  on 
it  will  show  a  motion 
video  sequence  from 
the  videodisc.  A  red 
icon  will  show  a  still 
frame. 

"TURN  OFF  VIDEO 
STILL"  gives  the  user 
control  of  how  long  the 
video  still  image  is  on 
the  screen 


Infformatlosi  Pop-up  Brattons 
Click  on  any  of  these  buttons  to  read  more  about  its 
contents.  When  finished  reading,  click  on  the  upper 
left  corner  of  the  pop-up  box,  and  it  will  go  away. 
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Sample  Quiz  Item 
DRAWINGS 


Quiz  Identification 
This  section  confirms  the 
chapter  and  quiz  type 
selected. 


Response  Selection  Buttons 
Click  on  box  A,  B,  C,  or  D  to  record  your  response. 
A  box  will  pop-up  in  center  screen,  to  tell  you  if  you 
are  correct  or  not. 


Stop  Quiz  Button 
Clicking  here  will  stop  the 
quiz  in  progress,  and  return 
the  user  to  the  PREFACE 
page. 
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A  SYSTEMS  APPROACH  TO  AUTOMATED  TECHNICAL  MANUAL  DEVELOPMENT 

INTRODUCTION 


As  tactical  systems  become  more  complex  the  paper  technical  documentation  created  to  support  them  has  grown 
in  size  and  complexity.  As  a  consequence,  system  operators  and  maintainers  are  faced  with  documentation 
storage  problems,  and  a  technical  manual  medium  which  has  inherent  limitations  for  accessing  relevant 
information.  This  is  especially  true  when  using  multiple  volume  technical  manuals. 

Recent  advances  in  microprocessors,  using  rapid  indexir>g  response  time  and  mass  memory  (approaching  the  1 
gigabyte  threshold),  have  led  to  plausible,  commercially  available  portable  devices.  In  conjunction,  the  ability  to 
apply  "Expert  System  Programming"  to  the  application  software  of  an  automated  technical  manual  provides  an 
irK:rease  in  user  efficiency.  This  is  achieved  by  faster  access  arxJ  data  retrieval  time.  Intelligently  hypertexted 
automated  techrucal  manuals  employ  hypermedia  principles  to  create  workpackages  that  link  specific  sections  of 
the  technical  manual  in  the  sequer^e  needed  to  correct  maintenance  problems  or  refresh  knowledge  of  system 
operating  procedures.  The  automated  technical  manual  developers  goal  is  to  increase  a  user’s  technical  abilities 
by  “workpackaging”  task  oriented  information  within  the  technical  manual.  A  workpackage  allows  the  user  to 
retrieve  related  data  elements  across  technical  manual  volumes,  in  a  prescribed  sequence,  to  accomplish  a  task. 
Workpackages  are  at  the  core  of  Automated  Technical  Manual  Functionality 

DRIVE  TO  PAPERLESS 


Space  and  weight  have  become  nrtore  important  as  we  push  for  more  capability  on  smaller  platforms.  Paper, 
especially  paper  technical  documentation,  has  long  been  targeted  as  a  major  space  and  weight  driver.  In  1987,  GE 
proposed  a  completely  paperless  technical  manual  for  the  AN/BSY-2  Submarine  Combat  System,  in  response  to 
the  Navy's  overwhelming  desire  to  save  space  on  the  SSN-21  Class  submarine.  The  BSY-2  is  the  largest,  most 
highly  integrated  submarine  combat  system  to  date,  and  the  estimated  size  of  its  technical  manual  is  50,000 
pages.  These  50,000  pages  are  roughly  split  between  text  and  graphics.  The  Navy  accepted  our  paperless 
approach,  yet  still  required  a  paper  manual  to  ease  the  transition  from  paper  to  paperless. 

Because  no  automated  technical  manual  specifications  existed  in  1987,  the  BSY-2  contract  did  not  contain  specific 
requirements  nor  specrTications  for  the  automated  technical  manual.  Early  in  the  program,  GE  agreed  to  write  a 
system  specification  based  on  three  high-level  requirements. 

a.  The  end-item  device  must  be  portable,  weigh  less,  and  consume  less  space  than  paper. 

b.  The  paperless  manual  must  provide  all  users  vwth  faster  information  access  when  compared  to  paper. 

c.  The  information  in  the  paperless  manual  must  be  identical  to  information  in  the  paper  version. 

With  these  three  top  level  requirements  GE  developed  paperless  manual  specifications  in  accordance  with  what 
we  now  refer  to  as  a  "systems  approach". 

THE  SYSTEMS  APPROACH 
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The  systems  approach  came  about  because  of  our  desire  to  satisfy  the  unique  needs  of  many  customers  and 
accomn^date  a  wide  rar^e  of  maintenance  and  operational  cortcepts,  yet  still  have  one  basic  process  for  creating 
an  automated  technical  manual  (ATM).  The  desire  to  have  one  basic  process  for  creating  an  ATM  is  driven  by 
ecoiTomic  considerations  and  a  belief  that  sinr^l®,  straight  forward  solutions  are  better  than  elaborate  custom 
solutions.  The  desire  to  accommodate  a  wide  range  of  maintenartce  and  operational  concepts  is  driven  by  the 
diversity  of  products  GE  buiW,  documents  and  maintains.  Our  goal  was  to  fieki  a  prw:ess  that  is  easy  and 
inexpensive  to  duplicate,  would  woiit^  equally  well  tor  all  products,  and  not  be  constrained  by  obsolete  technologies. 


With  the  systems  abroach  w®  identify  custoritar  needs  ar^  define  how  the  ATM  .will  be  used  to  support  a  system, 
up  front.  Then  we  implement  a  solution  to  achieve  it.  The  solution  considers  the  total  integrated  togistics  support 
picture  for  the  tactical  platform.  It  Inoirporates  the  maintenance  concept,  operational  requirermnts,  and  training 
needs  for  the  life-cycle  of  the  tactical  system.  With  the  systerm  af^oach,  technical  logistics  data  and  the  ATM 
hardware  and  software  are  integrated  to  meet  ILS  and  user  requirements.  This  is  accomplished  by  systematically 
analyzir^  user  requirennants,  forming  cortceptual  requirements,  refining  concepts  into  functional  requirements,  and 
develo^reg  system  requirements  that  define  how  to  tag  data  so  it  can  be  subsequently  linked  into  work  packages 
that  supi^rt  operational  artd  maintenartce  ransepts.  This  approach  works  equally  well  with  new  and  existing 
technical  manual  data. 

IB^PLEMENTATSOM  OF  THE  SYSTEM'S  APPROACH 


One®  the  systems  abroach  has  defined  ATM  fursslionality  and  how  it  will  access  information,  it  is  necessary  to 
author  and  tag  the  data.  The  tag  set  is  corrposed  of  SGML  tags.  Creators  of  the  automated  manual  must  be 
subject  nratter  experts  (SMEs)  on  tactical  system  operations/maintenance,  as  well  as  being  fanraliar  with  how  the 
automated  manual  will  be  used.  These  subject  matter  experts  embed  their  understanding  of  maintenance  and 
operations  into  maintenance,  operations  ar^  trairang  routines  using  SGML  tagged  data.  We  call  these  routines 
“wortqDackages".  The  ATM  developers  goal  is  to  increase  a  user’s  technical  abilities  by  the  workpackaging  of  task 
oriented  information  within  the  techrtical  manual.  A  worf^ckage  allows  the  user  to  retrieve  related  data  elements 
across  techr^cal  manual  volumes,  in  a  prescribed  ssquerKse,  to  accomplish  a  task.  The  sequence  is  defined  by  the 
subject  matter  expert  using  personal  experiesice  as  well  as  irtputs  from  Fleet  personnel.  The  development  of 
workpackagss  is  the  core  of  functionality  of  an  automated  technical  manual.  It  is  the  difference  between  fielding  an 
“electronic  page  turner”  or  a  true  expert-system  operation,  maintenance  and  training  aid. 

SMEs  rreist  be  technically  profident  in  txsth  the  content  of  the  technical  manual  and  how  it  is  to  be  presented;  so 
that  the  manual  iinfparts  the  greatest  benefit  to  the  user.  Data  elements  (paragraphs,  figures  and  tables)  must  be 
able  to  serve  nurrorous  woikpackages.  Knowledge  of  the  tactical  system  comWned  with  knowledge  of  the 
application  software  is  the  only  way  in  whidi  SMEs  can  effectively  create  workpackages.  Often,  SMEs  have 
performed  the  ta^f^  they  are  representing  through  workpackage  construction.  At  minimum,  workpackages 
should  indude  maintenance  and  operational  tasking.  Trainir^  routines  to  support  this  tasking  along  with 
presenting  basic  knowledge  of  the  system  might  also  be  included.  In  this  way  the  manual  can  serve  its  purpose  as 
a  reference,  problem  solving,  and  training  aid. 

WORKPACKAGE  UTILEATIOM 
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Workpackages  piovicie  users  with  quick  access  to  acojrate  data,  helpir^  them  to  solve  problems  faster.  Because 
subject  matter  experts  have  already  tagged  task  specific  data,  the  need  for  users  to  leaf  through  paper  manuals  to 
solve  maintenance  problems  or  find  operating  procedures  is  eliminated. 

The  BSY-2  system,  ike  mo«  modem  systems,  has  built-in  performance  monitoring  and  fault  localization 
capabiities.  When  a  failure  occurs,  an  operator/maintainar  is  alerted  with  a  fault  code.  To  fix  the  failure,  the 
maintainer  enters  the  fault  code  into  the  automated  technical  manual  directly,  instead  of  fumbing  through  paper 
manuals  in  search  of  a  fault  code  index.  The  fault  code  is  hypertext  inked  to  failure  specific  technical  data,  such  as 
remove/replace  procedures,  functional  block  diagrams,  circuit  diagrams  and  troubleshooting  routines.  Procedures 
for  these  are  suppled  step  by  step  with  highighted  warnings  and  cautions,  tools  and  parts  ists  and  hypertext  links 
to  other  appicable  procedures.  This  helps  ensure  both  thoroughness  and  accuracy  in  the  performance  of  the 
maintainer. 

Most  miitary  corrective  maintenance  concepts  require  a  maintainer  to  locaRze  and  then  rerTX>ve  and  replace  failed 
lowest  replaceable  units  (LRU).  If  replacing  the  LRU  does  not  solve  the  problem,  the  maintainer  is  automatically 
guided  through  more  detailed  documentation  to  continue  troubleshooting  the  problem.  Instead  of  having  to  sift 
through  several  paper  volumes,  the  maintainer  is  suppled  with  additional  reference  material  that  is  in  the  same 
wotkpackage  format  as  the  remove  and  replace  procedure.  The  subject  matter  expert  who  created  the 
workpackage  drew  on  design  experience  arid  linked  all  related  maintenance  documentation  in  the  workpackage  for 
the  maintainer.  By  simply  entering  one  fault  code,  the  maintainer  has  immediate  access  to  all  relevant  data 
needed  to  solve  Ns  maintenance  problem. 

Operator  workpackages  function  in  the  same  way  as  maintenance  workpackages.  Operator  workpackages  are 
primarily  used  to  refresh  knowledge  of  specific  operational  procedures.  Full  understanding  of  functions,  rrrodes 
and  system  capabiities  are  also  of  great  importance  to  operators.  These  elements  of  the  techrecal  documentation, 
as  well  depictions  of  displays,  controls  and  irKicators  are  all  inked  in  operator  workpackages.  The  operator 
workpackages  are  organized  by  functional  watchstation.  The  benefit  of  operator  workpackages  is  that  they  make  it 
easy  for  operators  to  keep  their  system  operational  skills  sharp. 

Supervisor  datapackages  are  orgaNzed  and  implemented  in  much  the  same  way  as  operator  and  maintenance 
workpackages.  However,  supervisors  need  have  access  to  data  which  relates  to  coordination  or  direction  of 
system  operation,  pravertative  maintenance,  and  corrective  maintenance.  These  datapackages  incorporate 
general  systems  knowledge  for  those  who  may  not  have  direct  system  responsibilities,  but  still  require  information 
to  support  traiNng.  Datapackages  are  not  procedural  in  nature  but  informational.  Datapackages  are  organized 
and  inked  by  topic.  These  datapackages  can  also  be  used  for  watchstation  qualification. 

TraiNng  workpackages  are  the  most  uNque  of  the  workpackages.  They  provide  embedded  on-board-training  of 
the  tactical  system  using  the  technical  manual.  Subject  matter  experts,  when  composing  the  manual  link  together 
technical  manual  elements  which  meet  tactical  system  traiNng  learNng  objectives.  TraiNng  workpackages  are 
used  by  instructors  to  encourage  arrd  improve  the  quality  of  student  self  study.  After  traiNng,  these  workpackages 
are  used  to  refresh  and  reirtorce  what  they  learned  in  the  classroom.  This  process  is  enhanced  using  pop-up 
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windows  containing  instructor  comments,  as  they  would  appear  in  an  instructor’s  guide.  At  the  end  of  each  training 
workpaAage  the  user  is  supplied  with  a  hypertexted  test  reviewing  the  workpackage.  If  the  user  answers  the 
question  correctly  he  is  allowed  to  proceed  to  the  next  question.  If  the  user  answers  the  test  question  incorrectly, 
the  user  is  hy^rtexted  to  the  point  in  the  manual  which  addresses  the  question.  In  this  way  users  can  receive 


Artother  feature  useful  for  training  is  the  session  replay  mode.  In  the  shore-based  training  environment,  a 
sigrtificant  gtortion  of  the  training  pipeline  is  dedicated  to  student  fanmliarization  of  the  organization  and  use  of  the 
technical  manual  contents.  This  contribiites  to  the  length  of  the  training  course  arxJ  its  associated  costs.  In  the 
session  replay  nrade  troubleshooting  problem-solvir^  routines  which  are  performed  by  the  student  can  be  stored 
on  the  automated  technical  manual.  The  instructor  can  then  “play  bad<"  the  technical  manual  references  the 
student  has  used  for  a  date-step  by  date-step  review  of  the  student’s  logic  (an  instant  replay).  This  feature 
provides  insight  into  the  student's  methoctoh^y  for  the  instructor  and  immediate  feedback  to  the  student. 


Other  modes  of  operation,  which  supply  user  a^ss  in  a  more  conventional  fashion  are; 

®  The  main  menu  contains  those  nrtodes  of  operafion  available  to  the  user.  The  user  selects  the  desired  mode, 
the  main  ntertu  bar  (which  allows  user  selections)  changes  relative  to  the  mode  of  operation  selected. 


The  table  of  contents  allows  the  user  to  view  the  table  of  cor^ents,  list  of  figures  and  list  of  tables  per  volume  or 
for  the  entire  technical  manual.  The  technical  manual  elenrtents  are  initially  presented  at  a  high  level,  volume, 
or  chapter.  These  entries  can  be  expanded  by  the  user  to  include  lower  level  paragraphs,  figures,  illustrations. 
All  of  the  table  of  contents  entries  are  hypertext  linked  to  the  appropriate  manual  element. 


•>  The  index  fursctions  allow  users  to  view  a  list  of  topics  for  each  specific  volume.  Each  of  these  elenrjents  is 
hypertext  Knksd  to  their  appropriate  manual  element.  The  search  function  allows  the  user  to  type  a  character 
string  and  search  through  the  full  text  or  table  of  contents  to  find  all  occurrences  of  the  string.  This  feature  is 
available  in  all  modes  of  operation. 


“  The  brows®  rrtode  allows  the  user  to  access  any  volume  of  the  technical  manual  in  a  free  manner.  The  user 
turns  pages  electronically  until  the  pwnt  of  interest  is  located.  This  mode  functions  like  that  of  a  paper  text,  only 
faster  using  hyf^rtext  links  and  the  search  fundion. 

PAPERLESS  MAf^UAL  PRODUCTION  PROCESS 


Our  experieixe  shows  that  the  quid^est,  least  costly  way  to  produce  an  automated  technical  manual  is  to  utilize  an 
electronic  pxjblishing  system  which  prowdes  a  writer  frierKlly  environment  where  authoring,  changes  and  revisions 
can  easily  a^mplished.  The  resultir^  technical  manual  data  is  stored  as  neutral  ASCII  SGML  ta^ed  files. 
The  ideal  system  would  be  designed  to  output  several  mediums  arKf  be  non-proprietary  in  its  corrponents. 

SGML  tags  serve  two  purposes:  they  identify  content  and  serve  as  workpackagirrg  building  blocks.  The  tags  are 
translated  or  igrwred  by  both  the  paper  production  process  and  the  hypermedia  software,  depending  on  the  tag. 
This  structuring  results  in  one  production  process  which  can  output  multiple  media  products.  Paperless  manual 
prafucers  shoukJ  also  be  concerned  with  the  limiting  aspect  of  proprietary  data.  ASCII  files  are  understood  by  all 
publishing  and  hypermedia  tools.  It  is  a  universal  standard  for  character  encoding.  SGML  tagging  is  a  neutral  and 
fxn-proprietary  method  to  distinguish  between  the  hypertext  tags  and  data  or  content  in  the  ASCII  files. 

Graphics  are  a  large  part  of  any  technical  manual  and  ^se  special  consideration.  There  are  basically  two  distinct 
sources  of  graphics  for  technical  manual  producers;  engineeiir>g  drawings  and  drawings  aeated  solely  for  use  in 
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the  technical  manual.  All  technical  manual  images  should  be  stored  in  a  graphics  data  base,  in  a  form  which  can 
be  processed  to  provide  either  paper  (postscript)  or  electronic  (raster)  output.  It  is  not  necessary  that  the  input 
image  format  is  the  same,  as  long  as  its  format  can  provide  both  outputs. 

OrK%  text  and  graphic  format  arxl  storage  problerrs  are  solved,  text  and  graphics  need  to  be  linked  together  by 
SGML  tags  in  the  text  and  hypertext  links  attached  to  the  graphic  image.  SGML  tags  in  the  text  are  used  as 
pointers  to  indicate  that  a  graphic  image  should  be  Inserted  at  specified  points  in  the  text.  The  SGML  tags  also 
provide  an  address  for  text  refererx^s  to  lirtk  to  the  image  location.  Hypertext  links  are  placed  on  the  image  and 
Snked  to  textual  elements.  All  linking  is  done  through  the  use  of  unique  syn*olic  references  which  are  given  a 
physical  address  only  when  the  data  is  translated  for  the  electronic  merSum.  This  allows  data  to  be  inserted  and 
deleted  without  requiring  re-authoring  of  the  links. 

Configuration  of  the  technical  manual  database  must  also  be  considered.  The  organization  of  a  tech  marujal  is  a 
heirarchy  of  volumes,  parts,  chapters,  paragraphs,  figures,  tables,  and  steps.  For  example,  we  are  grouping  in  a 
directory  structure,  all  chapters  in  a  volume.  Chapters  are  stored  as  a  file.  Numbered  paragraphs  are  identified  by 
a  cross-referorx5e  tags  which  contain  a  unique  symboBc  name.  Tables  and  figures  are  identified  in  the  same 
manner.  It  is  possible  to  extend  this  method  to  steps,  warnings,  cautions  and  notes.  The  symbolic  name  is  used 
by  authors  to  cross-reference  elemerrts  in  the  technical  manual  text.  It  is  also  used  by  subject  matter  experts  to 
script  work  packages. 

END  ITEM  CONSIDERATIONS 


We  call  the  end-item  device  for  the  automated  technical  manual  an  Alternate  Media  Display  (AMD).  The 
requirements  for  AMD  selection  are  identified  and  analyzed  using  the  systems  approach  which  ensures  that  end 
users  and  their  needs  are  fuHy  considered.  The  results  of  this  analysis  are  translated  into  functional  requirements 
that  are  then  used  to  perform  a  technology  search  to  find  candidate  AMDs.  The  AMD  candidate  that  meets  the 
largest  number  of  requirements  is  selected.  The  tactical  environment  will  cSctate  the  degree  of  mggedization 
required  by  the  AMD.  For  example,  the  submarine  environment  is  not  nearly  as  hazardous,  on  portable  electronic 
equipment,  as  a  earner  flight  deck.  Therefore,  It  may  be  more  cost  effective  to  purchase  a  Commerdal-Off-The- 
Shelf  (COTS)  device  rather  than  investing  in  mggedization. 

AMD  software  must  provide  the  data  retrieval  function  and  complement  the  AMD’s  marr-machine  Interface. 
Several  COTS  products  are  available  that  provide  this  capability.  Many  products  provide  browsing  functions  that 
allow  the  user  access  to  data  through  Table  of  Contents,  Index,  and  menu  functions.  Many  of  them  also  provide 
some  capability  to  Ink  cross  referenced  data  with  hypertext.  We  chose  a  combination  of  two  software  programs 
that  work  together  to  provide  these  functions  and  allow  us  to  easily  construct  workpackages. 

In  creating  an  AMD  one  should  avoid  customized  and  project-  unique  solutions  because  they  are  costly  and 
constrain  you  to  a  single  techrwiogical  solution.  On  the  BSY-2  we  decided  to  go  vrith  COTS  products  because  they 
are  inexpensive,  modular,  arxj  allow  customization.  By  doing  this,  we  developed  a  flexible  automated  technical 
manual  system.  Modular  COTS  products  can  be  replaced,  if  required,  to  meet  unique  project  needs.  Most 
importantly  this  process  allows  ATM  developers  to  float  with  technological  advances!  This  ensures  that  the 
automated  technical  manual  uses  the  most  current  technologies  throughout  the  life-cycle  of  the  tactical  system. 
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COWCLUSSON 


TechroSogy  tor  creating  and  fielding  automated  technical  marruals  is  here  today  and  it  makes  sense  to  apply  this 
techr^otogy  on  large  systerre  having  multiple  volume  technical  volumes.  The  key  to  cos!  effective  implementation 
is  to  use  a  systems  approach  arto  COTS  products.  The  systems  approach  reduces  development  risk  by 
integratir^  operating  arto  mairtenar^ce  concepts  with  user  needs  to  ensure  that  what  you  field  is  what  is  needed. 
COTS  hardware  arKf  software  lets  you  float  with  technology  until  it  is  time  to  actually  field  the  automated  technical 
manual. 

Creating  an  Automated  Technical  Manual  is  easily  within  the  capabilities  of  most  contractors  and  provides  a  large, 
measurable  ircreas®  in  operator  and  maintainer  effectiveness. 
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-ABSTRACT- 


A  PROTOTYPING  SYSTEM  FOR 
SIMULATION  ENTITY  INTERACTIVE  BEHAVIOR  CONTROL 

Across  the  DoD  weapon  systems  engineering  and  tactical 
training  communities  there  is  a  joint  requirement  to  simultaneously 
reduce  the  cost  and  increase  the  realism  of  interoperability 
training  exercises .  One  approach  to  achieving  this  joint 
requirement  is  to  replace  specific  subsets  of  the  manned 
interoperational  contingent  (operational  platforms  or  manned 
trainer  ("player")  stations)  with  computer-controlled  entity 
simulations . 

To  date  three  techniques  have  been  used  to  implement  computer- 
controlled  entity  simulation  in  an  attempt  to  reduce  the  number  of 
manned  trainer  "player"  stations; 

o  Instructor-Controlled  Simulation 

o  "Semi -Automated  Forces"  (SAFOR) -Controlled  Simulation 

o  Autonomously-Controlled  ("Intelligent")  Simulation 

Each  of  these  three  simulation  techniques  has  shown  varying  degrees 
of  benefit  depending  on  the  operational  dynamics  and/or  behaviors 
required  for  training. 

This  paper  presents  a  design  concept  for  a  fourth  technique 
for  computer-controlled  entity  simulation:  that  of  Supervisor- 
Controlled  Simulation.  Supervisor-Controlled  Simulation  places  the 
human  in  the  role  of  a  supervisor  of  semi-autonomously-controlled 
simulation  entities.  As  a  supervisor,  the  human  is  able  to 
intervene  in  the  otherwise  computer-controlled  deductive  reasoning 
and  procedure  control  processes  to  assist  the  subordinate  in 
situation  assessment  and  tactical  goal  selection.  The  human 
intervention  enables  a  degree  of  inductive  reasoning  and  human 
"unpredictability"  to  be  injected  in  the  simulated  entity  behavior. 
This  paper  also  presents  the  requirements  and  top-level  components 
for  a  Supervisory  Control  simulation  test-bed  called  the 
Interactive  Behavior  Control  System  (IBCS).  The  supervisory 
control  simulation  test-bed  must  be  developed  to  enable  the 
evaluation  of  alternative  concepts  for  "leveraging"  human  reasoning 
across  a  large  and  potentially  diverse  set  of  otherwise  computer- 
controlled  simulation  entities. 

The  Supervisory-Controlled  Simulation  design  is  being  pursued 
in  a  multi-year  IRAD  program  by  Ball  Systems  Engineering 
Division.  The  concept  of  supervisory  control  simulation  is 
beneficial  for  individual  or  team  training  simulation  systems 
configurations.  The  supervisory-control  concept  provides  a 
"sliding  scale"  of  simulated  entity  control:  from  complete  control 
by  the  instructor  to  f ully-autonomous  interaction. 


DESIGJI  REQUIREMEHTS 


The  top-level  requirements  for  the  IBCS  concept  fall  into  two 
categories;  (1)  functional  and  (2)  computational.  The  functional 
requirements  for  the  IBCS  concept  include  the  following; 

o  Provide  realistic  command  and  control  dynamics  simulation 
for  all  critical  training  cues  associated  with  all 
classes  of  semi-automated  simulation  entities, 
o  Provide  for  supervisory  intervention  in  any  of  the 
entity's  on-going  command  and  control  processes, 
o  Maintain  simulated  entity  real-time  performance  (both  in 
decision-making  and  systems  control  directives)  for  all 
the  simulation  entities  under  the  IBCS  control, 
o  Avoid  unrealistic/extra  "Situation  Awareness"  to  be 

available  to  the  supervisor  for  any  simulation  entity. 

The  computational  requirements  for  the  IBCS  concept  include  the 
following; 

o  Support  a  hybrid  of  simulation  entity  decision-support 
software  including;  (a)  decision-theoretic  and/or 
utility  theory  algorithms;  (b)  real-time  expert  systems; 
(c)  neural  networks;  and  (d)  mathematical  programming, 
o  Support  training  instructor  direct  simulation  entity 

control  modes. 

o  Support  an  object-oriented  interface  to  the  simulation 
entity's  subsystem  control  simulation  software, 
o  Support  distributed  interactive  network  interfaces . 

The  level  of  realism  required  for  computer-controlled  threat 
simulation  is  driven  by  the  operational  accuracy  with  which 
tactical-decision-making  "cues"  must  be  statically  and  dynamically 
represented  to  the  trainee.  The  simulation  realism  required  for 
realistic  cues  is  a  function  of  the  sensitivity/observability  of 
the  trainee's  on-board  sensors  and  communication  systems  as  well  as 
the  expected  interaction  dynamics  that  operational  players  and 
simulation  systems  can  have  within  large-scale  interoperability 
simulation  training  exercises.  Table  1  characterizes  four 
"interaction  states"  that  differentiate  the  simulation  entity 
control  systems  that  could  be  expected  to  provide  "cue  realism"  to 
the  trainee  positions  on  the  simulation  network.  IBCS  should  be 
configured  to  control  all  entity  interactions  with  the  trainee  that 
do  not  require  continuous  explicit  communication  and/or  "ACM-level" 
operator  reasoning. 

The  IBCS  must  be  designed  to  interact  with  the  manned 
positions  on  a  distributed  interactive  network.  The  networking 
approach  to  distributed  interoperability  training  can  be  expressed 
in  simplified  fashion  by  the  diagram  in  Figure  1.  Figure  1 
displays  three  categories  of  networked  systems:  primary  training 
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systems,  secondary  training  systems,  and  supplemental  training 
systems.  The  primary  and  secondary  training  systems  are  defined  to 
be  "manned  systems"  that  may  be  linked  to  each  other  by  data  links 
(simulated  or  operational)  and/or  communication  channels.  The 
supplemental  training  systems  provide  additional  interoperability 
realism  and  can  be  made  up  of  a  hybrid  of  manned  and  unmanned 
simulation  entity  controllers. 

The  supplemental  systems  can  be  further  partitioned  into  three 
categories:  (1)  Semi -Autonomous  Forces  (SAFOR)  control  simulation 
systems;  (2)  Supervisory  control  simulation  systems;  and  (3) 
Autonomous  control  simulation  systems.  The  semi-intelligent 
simulation  entities  associated  with  the  SAFOR  commander's  position 
must  be  functionally  responsive  to  the  SAFOR  commander's  simulation 
state  dynamics  (see  References  1  and  2).  The  semi-intelligent 
simulation  entities  associated  with  the  IBCS  concept  are  responsive 
to  a  combination  of  the  supervisor's  directives  and  computer¬ 
generated  procedures  directives.  The  difference  between  SAFOR 
control  and  Supervisory  control  is  that  the  supervisor  is  not  a 
direct  war-fighting  participant  and  is  "free"  to  intervene  across 
a  spectrum  of  simulation  entities,  where  as,  the  SAFOR  commander  is 
a  war-fighting  participant  and  is  obligated  to  man  his  position, 
thereby,  reducing  the  SAFOR  Commander's  capacity  (during  high  task¬ 
loading  situations)  to  provide  explicit  semi-autonomous  control 
intervention . 


TABLE  1  -  TRAINEE /ENTITY  SIMULATION  CONTROL  STATES 


TRAINEE /THREAT  INTERACTION 
STATE 

SIMULATION  ENTITY  CONTROL 
TECHNIQUE 

Threat  ACM  or  Close  Formation 
Communication  and  Coordination 

Individual  MIL*  Simulation 
Stations  Required 

Secondary  Player  to  the 
Primary  Team  C3I  Interaction 

Low  fidelity  MIL  and/or  Semi- 
Autonomous  Simulations 

Entities  Inside  the  Ownship's 
Immediate  Offensive  or 
Defensive  Planning  Envelope 

S emi -Au t o nomo u s 
Simulations 

Entities  Outside  the  Ownship's 
Immediate  Offensive  or 
Defensive  Planning  Envelope 

Autonomous 

Simulations 

*  MIL  -  Man-in-the-loop;  Semi -Autonomous  -  human-assisted 
computer-controlled  simulations;  Autonomous  -  no  human  assistance 
in  the  entity  control  simulation 


To  have  the  greatest  utility,  the  IBCS  concept  must  free-up 
the  instructor  while  at  the  same  time  providing  a  degree  of  human¬ 
like  behavior  to  a  set  of  entities  that  are  most  critical  to  the 
trainee's  tasking.  The  human-like  behavior  should  be  achieved  by 
a  blend  of  autonomous  expert-systems-like  procedure  controllers  and 
aperiodic  human  reasoning  (see  reference  3),  The  overall  IBCS 
control  concept  must  be  partitioned  to  enable  model  validation  and 
certification  in  the  context  behavioral-associated  software 
processes  (perception,  reasoning,  decision-making,  and  procedure 
control)  (see  Reference  4).  Lastly,  the  IBCS  concept  must  embody 
the  principles  of  supervisory  control  (see  Reference  5).  The  key 
design  trade-off  is  the  sophistication  of  autonomous  deductive 
reasoning  systems  and  the  design  of  a  real-time  human-to-expert 
systems  interface « 

To  be  effective,  the  supervisory  control  interface  must 
support  three  primary  processes:  (1)  supervisor's  receipt  of 
subordinate  requests  for  advice  and  direction;  (2)  subordinate's 
receipt  of  supervisor's  advice  and  direction  (provided  in  a  timely 
and  contextually-meaningful  set  of  goals  and  objectives);  and  (3) 
a  subordinate  activity  simulation  that  transforms  the  goals  and 
objectives  into  explicit  systems  procedural  coiimand  and  control 
activity  as  a  function  of  both  the  immediately  perceived  situation 
and  the  goals  and  objectives  (see  Reference  5), 


FIGURE  1  -  DISTRIBUTED  SIMULATION  CONTROL  COMPONENTS 
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CONCEPT  OVERVIEW 


The  IBCS  design  is  comprised  of  three  simulation  processes: 

•  World  State  Update  Simulation 

•  Human  Activity  Simulation 

"  Human  Intervention  Simulation 

The  goal  of  the  supervisory  control  mode  is  to  provide  timely 
management  directives  where  appropriate  while  avoiding  the 
requirement  for  micro-management.  The  appropriate  stages  of 
intervention  will  be  determined  by  "assessing"  the  need  (utility) 
to  apply  human  inductive  reasoning  to  assist  the  subordinate  human 
activity  simulation  in  making  "realistic"  decisions.  The  supervisor 
will  be  prompted  to  intervene  within  the  simulated  entity's 
decision-making  state  in  an  operationally-consistent  manner.  The 
"options"  for  and  the  timing  of  these  interventions  will  be  under 
the  joint  control  of  the  computer-based  human  activity  simulation 
and  the  human  intervention  simulation.  Figure  2  provides  an 
overview  of  the  critical  information  flow  between  the  primary 
simulations  that  comprise  the  IBCS. 


The  cornerstone  of  our  approach  is  the  co-execution  of 
supervisory  control  simulation  and  human  activity  simulation 
modules.  The  supervisor  is  afforded  direct  access  to  the  human 
activity  simulation  module  processing  and  the  human  activity 
simulation  is  jointly  responsive  to  the  supervisor's  directives  as 
well  as  its  own  dynamic  assessment  and  control  processes. 
Providing  for  supervisory  control  means  that  the  embedded  expert 
systems  that  control  the  simulated  human  activity  can  be  oriented 
toward  the  deductive  behavior  processes  while  the  supervisor  can  be 
prompted  to  intervene  in  the  human  activity  simulation  to  interject 
inductive  reasoning.  The  separation  of  the  inductive  and  deductive 
reasoning  processes  is  a  critical  aspect  of  our  concept. 


The  IBCS  will  develop  and  display  "intervention  cues"  and 
intervention  "state  data"  to  the  supervisor  via  the  Human 
Intervention  Simulation  (HIS).  The  intervention  state  data  will  be 
directly  accessible  and  aucrmentable  by  the  supervisor  for  any 
simulation  entity  whose  dynamic  behavior  is  projected  (via 
situation  uncertainty  (novelty)  detection)  to  benefit  from  the 
direct  application  of  human  judgement.  If  no  intervention  cue  is 


generated  and/or  no  intervention  action  is  initiated,  then  entity 
command  and  control  will  be  maintained  by  the  entity's  autonomous- 


control  simulation.  The  functional  interaction  between  the  primary 
computational  components  of  the  IBCS  processors  is  displayed  in 
Figure  3.  A  brief  description  of  these  IBCS  components  is  provided 
in  Table  2 . 
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FIGURE 


DIRECT 

HUMAN  EXPERT  EXPERT  SYSTEM  AND 

INTERVENTION  SYSTEMS  NEURAL  NETWORK 

SIMULATION  CONTROL  CONTROL 
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FIGURE  3  -  INTERACTIVE  BEHAVIOR  CONTROL  SYSTEM  FUNCTIONAL  FLOW 
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TABLE 


IBCS  COMPONENT  DESCRIPTIONS 


Human  Activity 
Simulation  (HAS) 

Simulation  of  effective  human  task 
activity  execution  based  on  human  multi¬ 
resource  supply  and  demand 

Human  Intervent. 

Simulation 

(HIS) 

Simulation  the  translates  the  current  HAS 
state  and  request  for  intervention  into 
"effective"  entity  Situation  Awareness 
for  the  supervisor.  HIS  also  provides  the 
supervisor's  controls /displays  interface 
simulation  to  enable  intervention  advice 
to  be  generated  in  terms  that  the  HAS  can 
interpret. 

Deductive  Agent 
Simulation  (DAS) 

Expert-systems-based  simulation  of  weapon 
system  response  selection.  Response  state 
is  a  function  of  the  assessed  operational 
situation  and  the  current  mission  state 
utility  of  the  alternative  weapon  systems 
responses . 

Procedure  Control 
Simulation  (PCS) 

Provides  simulated  weapon  system(s) 
control  update  for  procedural  tasking 
requiring  no  intervention  command/control 

World  State 

Update  Simulation 
(WSUS) 

Provides  for  all  simulation  entity  state 
updates  in  response  to  control  commands 
generated  by  HAS  and  PCS 

Effective  C&D 

State  Update 
Simulation  (ECDS) 

Manages  explicit  WSUS  command  generation 
and  generates  effective  situation  stimuli 
to  drive  the  HAS  model 

Perfoinnance 
Assessment  System 
(PAS) 

Develops  the  correlation  of  human  and 
computer-aided  situation  assessment  to 
simulation  entity  state  dynamics  repres . 

The  key  processes  of  the  IBCS  concept  are  the  Human  Activity 
Simulation  (HAS)  and  the  Human  Intervention  Simulation  (HIS).  The 
interaction  between  HAS  and  HIS  provide  the  human-like  behavior  to 
the  simulation  entities.  The  functional  HAS  processing  elements 
(shown  in  Figure  3)  are  described  in  Table  3.  A  critical  component 
of  the  HAS  is  the  Taskload  Processing  simulation.  The  Taskload 
Processing  simulation  uses  multi-resource  theory  to  constrain  the 
operator's  effective  task  completion  throughput  (see  Reference  6). 
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TABLE  3  -  HUMAN  ACTIVITY  SIMULATION  ELEMENTS  DEFINITION 


Situation 

Sensing 

Translates  situation  state  data  that 
would  be  available  to  the  operator  into 
operator-detected  state  data.  The  data 
is  formatted  into  situation  assessment 
data  for  use  by  the  Situation  Assessment 
process  as  well  as  the  HIS. 

Situation 

Assessment 

Simulates:  (1)  the  process  of  developing 
the  operator's  active  hypotheses  of  the 
current  mission  situation;  (2)  testing 
the  sensed  situation  data  against  the 
active  hypotheses;  and  (3)  selecting  an 
operative  hypothesis  based  on  confirming 
evidence.  This  process  is  augmentable 
by  the  HIS. 

Response 

Selection 

Simulates  the  decision-making  for  future 
weapon  systems  command  and  control  based 
on  the  currently-active  operative 
situation  hypotheses.  This  process 
evaluates  alternative  tactical  response 
options  and  selects  the  preferred  option 
for  execution  scheduling. 

Taskload 

Processing 

Simulates  the  operator's  explicit 
completion  rate  of  task  activity 
associated  with  the  operator's  required 
weapon  system  command  and  control 
processing. 

Operator 

Commands 

Generation 

Simulates  the  prioritization  and 
scheduling  of  weapon  systems  command  and 
control  activity  associated  with  the 
response  selections  from  the  Response 
Selection  processor.  Sets  database 
elements  associated  with  the  ECUS  and 

WSUS  interfaces. 

The  HIS  will  enable  a  human  (Supervisory  controller)  to 
directly  or  indirectly  intervene  in  the  on-going  computer-based 
command  and  control  process  at  critical  times.  Direct  intervention 
in  the  command  and  control  process  means  that  the  supervisory 
controller  would  be  provided  specif ic/timely  situation  stimuli  that 
would  be  operationally  present  (ie.  from  displays,  data  links,  and 
step-stare  out-the-window  scene  generation)  as  a  function  of  the 
simulated  operator's  on-board  "vision”  process.  Indirect 
intervention  in  the  command  and  control  process  means  that  the 
exercise  controller  would  receive  and  enable  or  augment  pre- 
processed  situation  assessment  state  data  (network  graphics 
depicting  both  current  and  historical  time/event  activity)  for  the 
intervened  entity.  HAS  will  continually  form  and  test  active 
situation  state  assessment  hypotheses  as  a  part  of  the  HAS 
Situation  Assessment  (SA)  processor.  If  the  supervisor  intervenes 
within  the  entity's  SA  process,  then  he  will  be  provided  both 
computer-based  active  and  operative  hypotheses  associated  with  the 
currently  prioritized  entity  command  and  control  state.  When  the 
human  does  not  involve  himself  in  the  command  and  control  process, 
the  HAS  will  provide  real-time  computer-based  intelligent  systems 
command  and  control  to  the  WSUS  to  "close-the-loop" .  The  WSUS 
component  of  IBCS  will  execute  the  control  commands  independent  of 
whether  these  commands  come  from  the  human  or  from  the  computer- 
based  intelligent  systems.  The  Human  Intervention  Interface 
processes  are  displayed  in  Figure  4 . 


HSDB  (HIGH  SPEED  DATA  BUS) 


HUMAN  i 
’  INTERVENTION 
INTERFACE  ! 


HUMAN 
ACnVITY  I 
SIMULATION  I 


FIGURE  4  -  HUMAK  INTERVENTION  SIMUUITION  PROCESSES 
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Figure  5  highlights  3  alternative  "states"  of  control  for 
which  the  IBCS  would  generate  3  different  levels  of  simulation 
processing.  The  activity  flow  associated  with  each  set  of 
simulation  processing  highlights  the  interaction  between  the 
primary  IBCS  components.  Figure  5  distinguishes  3  simulation 
"conditions"  where  different  states  of  human  intervention  can  be 
applied  to  increase  the  realism  of  the  simulation  entity's 
performance  and  appearance. 

PROTOTYPING  ACTIVITY 

Ball  Systems  Engineering  Division  is  developing  a  system  to 
prototype  the  IBCS  under  a  three  year  IRAD  program.  The  IRAD 
program  began  in  1991.  The  prototype  IBCS  will  be  developed  to 
enable  extensive  supervisory  control  mode  testing  in  a  stand-alone 
configuration  or  in  a  networked  system  with  other  distributed 
interactive  simulation  (DIS)  control  network  nodes  that  have  been 
developed.  Figure  6  diagr2uns  the  components  of  the  IBCS  in 
conjunction  with  other  Local  Area  Network  (LAN) -connected  DIS 
processors . 

Our  approach  toward  developing  the  IBCS  prototype  is  divided  into 
four  phases; 

o  Phase  1  -  Design  Requirements 

o  Phase  2  -  Core  Systems  Development  and  Demonstration 

o  Phase  3  -  Human  Intervention  Simulation  Development  and 

Integration 

o  Phase  4  -  Network  Interface  Development  and  Concept 
Verification  Testing 

The  total  IBCS  prototyping  program  will  be  structured  to 
configure  an  IBCS  demonstration  version  by  the  end  of  1992.  A 
user-verified  test  system  will  be  developed  by  1993.  The  mission 
domain  for  the  user-verified  test  system  will  be  a  Helicopter- 
Airborne  Forward  Area  Control  mission.  Enough  detail  regarding  the 
critical  mission  element  scenario  dynamics  will  be  simulated  to 
demonstrate  the  feasibility  and  flexibility  of  the  IBCS 
Supervisory-Controlled  Simulation  concept. 
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FIGURE  5  -  SAMPLE  INTERVENTION  STATES 
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RELEVAHCE  TO  OVERMX  TRAINING  RESEARCH  OBJECTIVES 


The  IBCS  concept  offers  the  potential  for  three  main  benefits: 
(1)  a  periodic  human-in-the-loop  performance  (explicit  allowance  of 
unpredictability  and  reasoning  under  uncertainty);  (2)  a  reduction 
in  required  complexity  for  expert  systems  entity  controllers  (by 
reducing  the  need  for  the  representation  of  uncertain  inductive 
reasoning  within  the  expert  systems);  and  (3)  a  simulation  testbed 
whose  architecture  will  permit  an  efficient  validation  process  (due 
to  a  behavior-theory-compliant  systems  architecture),, 

The  IBCS  will  complement  the  SAFOR  Commander's  Station  and 
enable  substantial  numbers  (lOO's)  of  intelligent  simulation 
entities  to  be  maintained  as  realistic  "players"  in  the  simulation 
for  extended  periods  of  contiguous  time/event  space  by  a  single 
supervisory  controller.  These  intelligent  players  may  even  be 
linked  via  the  currently-developed  SAFOR  command  and  control 
structures  to  create  a  multiplying  effect  for  intelligent  control 
of  up  to  1000 's  of  players.  The  implementation  of  "fast  moving 
platforms"  and  highly  dynamic  situation  assessment  processing  into 
the  distributed  interactive  simulation  training  domain  will  benefit 
from  the  IBCS  concept. 

Successful  development  of  the  IBCS  will  enable  the  derivation 
of  mission-segment  control  strategies  and  associated  data  bases 
that  maximize  the  set  of  computer-controlled  entities  that  can 
effectively  be  managed  by  a  single  human  ins tructor /operator . 
Properly  configured  and  dynamically  managed,  these  mission-segment- 
specific  databases  can  be  linked  during  interactive  simulation  to 
form  composite  human /computer-based  mission  level  command  and 
control  policy  sequences .  These  sequences  would  form  the  control 
strategy  database  for  the  Human  Intervention  Simulation  (HIS). 
Through  iterative  testing  and  database  maturation,  the  HIS  would  be 
reconfigured  in  conjunction  with  the  HAS  and  the  WSUS  to  form  the 
basis  of  training-application-specific  "Interactive  Adversary 
Control  Systems"  (lACS).  These  lACS  would  form  the  kernel  for  a 
large  set  of  simulation-centered  weapons  engineering  and  DIS 
systems  that  require  complex  large-scale  intelligent  adversary 
simulation  design  environments. 


146 


REFERENCES 


1.  "SIMNET  Semi -Automated  Forces  Version  3.8",  BBN  Report  No. 
DRAFT-7025,  19  January  1990. 

2.  "Replacing  the  Exercise  Controller  with  the  Enemy"  The  SIMNET 
Semi -Automated  Forces  Approach,  BBN  Report  No.  7211,  December 
1989. 

3.  "Realtime  Expert  Advisory  Systems;  Considerations  and 

Imperatives",  C.  Barrett,  M.  Donnell,  Information  and  Decision 
Technologies;  16  (1990)  pgs .  15-25 .hierarchical  decision¬ 

making  simulation  training  exercises. 

4  "Human  Performance  Models  for  Computer-Aided  Engineering  ,  J. 
Elkind,  S.  Card,  etal.,  Editors,  Committee  on  Human  Factors, 
National  Academy  Press,  Washington,  DC,  1989. 

5.  "Research  and  Modeling  of  Supervisory  Control  Behavior",  T. 
Sheridan,  R.  Hennessy,  Editors,  Committee  on  Human  Factors, 
National  Academy  Press,  Washington,  DC  1984. 

6.  "Combined  Mission/Taskload  Effectiveness  Model",  L.  Jobson, 
Victory  Integrated  Systems,  1989. 


147 


MISSION  SYSTEM  TRAINING  THROUGH  THE  NETWORKING 
OF  TWO  WEAPON  SYSTEM  TRAINERS  (U.S.  NAVY  DEVICE  2F146) 


A  Paper  for  the  Third  Annual  Airborne 
Weapons  Technology  Review  and  Training  Exposition 


Ms.  Claire  M.  Clinton  and  Dr.  C.  Michael  McCarthy 
Reflectone,  Inc. 

Tampa,  Florida 


ABSTRACT 


Linking  independent  task  trainers  together  fo  provide  complete  crew  training  has  become  a  growing  simulation  requirement.  The  SH-60F 
Weapon  System  Trainers  (WSTs)  allow  the  traditional  integration  of  an  independeru  Sensor  Operator  Trainer  (SOT)  with  an  independent 
Operational  Flight  Trainer  (OFT)  to  yield  a  Weapon  System  Trainer.  It  then  takes  this  training  scenario  one  step  further  by  networking  two  WSTs 
to  create  a  Mission  System  Trainer  (MSI)  to  enable  coordinated  training  for  two  anti-submarine  warfare  (ASW)  crews  in  a  common  tactical 
problem  world.  Within  such  an  environment,  the  effects  of  another  crew 's  actions  in  the  scenario  are  generated  by  each  crew  directly,  requiring 
no  simulation  on  the  pan  of  the  instructor. 

The  MST  mode  cf  operation  furnishes  advanced  training  capabilities  in  cdrcnfft  to  aircnfft  communication  for  areas  of  datalink,  VHF/UHF, 
clear  and  secure,  HF,  air  to  air  TACAK,  and  underwater  communication.  Acoustic  interference  can  be  created  with  bodi  WSTs  dipping  sorutr, 
as  well  as  having  multiple  sonobuoys  on  the  same  sonar  frequency.  Formation  flight  training  is  enhanced  by  die  unpredictability  of  the  other 
aircrcfl's  movements  and  the  availability  of full  collision  detection  between  the  two  helicopters.  Thdning  scenarios  in  the  MSI  mode  can  extend 
so  the  training  of  multiple  creva  in  coordinated  take  qffb  and  landings  from  a  common  parent  skip  arul  dte  synchronization  of  tactical  maneuvers 
in  an  ASW  search  viAin  the  inner  zone. 

The  mechanism  of  connecting  the  WSTs  is  a  unique  Ethernet  protocol  developed  by  Reflectone,  and  through  the  direct  connection  of  the 
WSTs  Tactical  Data  Systems  for  die  purpose  cf  datalink.  Althoughallfour  trainers  (2  OFTs  and  2  SOTs)  are  controlled  by  their  own  instructor 
station  in  independentmode,  MST  mode  allows  for  total  problem  control  from  either  WST  instructor  station.  From  die  instructor  station  selected 
as  the  command  station,  the  problem  world  (target  types  and  positions,  problem  world  limits,  weather,  ocean  type,  biologies,  etc.)  is  created 
in  the  same  manner  as  for  a  single  WTT,  except  that  now  it  is  made  common  to  boA  trainers.  The  individual  environment  of  die  trainer  such 
as  ordnance  load,  malfunctions,  and  fuel  quantities  is  still  maintained  by  means  of  a  helicopter  designator  Junction.  This  function  also  directs 
die  instructor  control  over  the  individual  trainers  in  such  areas  as  parameter  freezes,  slew  control,  motion  and  hydraulics. 

The  technical  modeling  implementation,  vAich  permits  these  trainers  to  grow from  a  single  task  trainer  to  a  complex  mission  trainer,  allows 
for  highly  diversified  training  capabilities.  In  this  way,  advanced  coordinated  ASW  crew  training,  previously  unobtainable  in  a  simulation 
environment,  it  now  a  viable  option. 


INTRODUCTION 

Increased  budget  constraints,  accompanied  by  the 
need  for  a  higher  level  of  training  in  conditions  which 
cannot  be  duplicated  in  peace  time  manuevers  and 
"team”  training  requirements  have  made  it  necessary 
to  develop  simulators  capable  of  rendering  more  than 
the  basic  training  they  have  traditionally  provided. 
Networking  single  task  trainers  into  mission  oriented 
training  systems  is  a  low  cost,  highly  effective 


method  of  acheiving  this  goal.  The  networking  of  a 
large  number  of  low  cost  trainers  into  a  common 
battlefield  for  ground  combat  training  has  been 
demonstrated  by  SIMNET  [1].  Air-to-air  combat 
training  is  available  in  simulation  through  the 
MULTISIM  program  at  Fort  Rucker  [2].  Now,  the 
networking  concept  has  been  applied  to  advanced 
coordinated  ASW  crew  training  in  the  SH-60F  (CV- 
Helo)  Mission  System  Trainer. 

The  CV-Helo  Weapon  System  Trainer’s 
requirement  is  "to  provide  the  functions  and 
equipment  necessary  to  train  the  Pilot,  Copilot  and 
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OFT  #1 


OFT  #2 


the  SH-60F’s  pnumy  missaoo  _ 

wa^a.,  effective  « 

would  not  be  acheived  in  the  trammg  of  a  smgl 
crew.  Consequently,  it  was  addition^ly  specili^ 
that  the  two  WST  trainers  have  the  add^  «q)a  ty 
TtetetLi  together  to  ereet.  «.  MST  trooer 

ttitie,  it  become.  p».ible  to  tj^ 
totogh  .imutalioii.  the  method,  of  duel  tem 

mckihg.  mid  prosecution  of  the  undentmer  tl^b 

of  the  duel  WST  intmuetton.  th.t  me 
JeSo  thi.  ost,  ere  function^  m  both  tnuners. 


XRAD'JER  layout 


The  complete  CV-Helo  simulator  system  consists 
of  trainers,  each  driven  by  their  own  comp^ 
id  ih  having  their  own 
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The  OFT  for  each  WST  is  a  simulated  cockpit 
with  an  eight  window/six  channel  visual  system, 
mounted  on  a  six-degree-of-freedom  motion  platform, 
capable  of  training  both  a  pilot  and  copilot.  The  fiilly 
equipped  onboard  instructor  station  is  used  for  the 
control  of  the  training  scenario  in  the  OFT 
independent  mode  only.  An  equivalent  floor 
instructor  station,  is  used  in  conjunction  with  the  SOT 
instructor  station  for  controlling  the  WST  scenario. 

The  SOT  trainee  station  is  a  fixed  base  shell 
containing  the  Tactical  Sensor  Operator  (TSO)  station 
and  the  Acoustic  Sensor  Operator  (ASO)  station.  The 
SOT  instructor  station  is  used  in  controlling  both  the 
independent  mode  and  the  WST  training  mode. 

In  WST  mode,  the  OFT  and  SOT  students  work 
together  as  in  the  real  aircraft.  Parameters  generated 
through  instructor  control  during  independent  mode 
are  now  replaced  by  the  actual  elements  resulting 
from  the  combined  crew’s  actions.  For  example, 
helicopter  position,  altitude,  and  speed  which  are 
instructor  input  variables  in  SOT  independent  mode 
are  now  determined  by  the  simulation  ouqiut  of  the 
OFT.  The  division  of  labor  between  the  OFT  and 
SOT  instructor  stations,  when  operating  as  a  WST 
instructor  station,  is  such  that  the  SOT  station 
controls  the  tactical  problem  world,  while  the  OFT 
station  commands  the  aircraft  and  its  environment. 
Parameters  which  are  selectable  on  both  trainers  such 
as  state  can  still  be  modified  from  either  station. 

MST  FEATURES 

The  MST  mode  is  simply  the  interaction  of  two 
WSTs  in  a  common  problem  world.  The  WST 
instructor  station  from  which  the  mode  is  requested 
hecomes  the  command  station,  and  is  now  capable  of 
controlling  both  WSTs.  The  key  feature  of  MST 
control  comes  from  the  fact  that  although  both  WSTs 
interact  in  the  same  problem  world,  governed  by  a 
global  freeze  and  a  single  initial  conttition,  the 
autonomy  of  eardi  helicopter  is  retained  by  the 
instructor  through  the  use  of  a  helicopter  designator 
function.  The  following  is  a  liat  of  features 
accomplished  from  a  single  command  station  which 
allow  changes  and  reconfiguration  of  one  WST 
without  interrupting  the  other  WST  when  in  the  MST 
mode: 

1)  Motion  and  hydraulic  systems  operate 
indqiendently  for  each  OFT. 

2)  Parameter  freezes  including  flight, 
position,  altitude,  speed  and  fuel  affect  each 
WST  individually. 


3)  Slew  functions  are  applied  selectively  to 
each  WST. 

4)  Separate  GCA/CCA/DCA  appraoch  plots 
can  be  obtained  for  both  WSTs. 

5)  System  malfunctions/degradations  and 
aircraft  emergencies  are  inserted  into  each 
WST  individually. 

6)  Helicopter  configurations,  including 
armament,  sonobouys,  and  sonar/cargo  hook 
are  managed  sqiarately. 

Although  the  above  controls  are  applied 
separately  to  the  individual  WST,  instructor  control 
of  the  tactical  problem  world  such  as  target  activation 
and  environmental  parameter  manipulation  need  only 
be  entered  once  to  be  applied  to  both  WSTs. 

The  MST  mode  of  operation  furnishes  improved 
training  capabilities  in  three  major  functional  areas: 

1)  Communications  :  The  instructor  is  able  to 

communicate  with  both  crews  through  the  ICS,  either 
selectively  by  trainee  position,  or  with  all  crew 
members  simultaneously.  WST  to  WST 
communication  may  be  accomplished  via  VHFAJHF 
normal  and  secure,  HF,  or  by  of  UQC 

(underwater  communications)  from  dipping  sonar  to 
dipping  sonar.  Range  and  bearing  calculations  are 
achievable  through  air  to  air  TACAN.  In  addition, 
the  Tactical  Data  Processor  Data  link  can  be 
acconqilished  between  WSTs. 

2)  Formation  Flight  :  Coordinated  flight, 
including  takeoffs  and  landings  from  a  common 
parent  ship  is  available  to  both  WSTs  in  the  MST 
mode.  aircraft  can  be  seen  in  the  other’s  visual 
scene  and  automatic  collision  detection  is  acheived 
through  the  sensing  of  the  intersection  of  the  two 
helicopter  volumes. 

3)  Acoustic  Effects  :  Sonobouys  launched  by 
either  helicopter  into  tiie  tactical  problem  world  can 
be  monitored  by  both  WSTs.  In  addition  to  the 
traditional  acoustic  interference  caused  by  two 
sonobouys  having  the  same  ensonifying  frequency,, 
the  MST  mode  also  allows  for  direct  path  acoustic 
interference  of  the  two  dipping  sonars. 
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DESIGN  CONSIDEllATIONS 

The  requiremefflts  for  the  integrated  modes  on 
these  trainers  demanded  several  special  design 
considerations.  The  data  being  conveyed  over  the 
Ethernet  must  be  transferred  at  a  high  enough  rate  to 
keep  the  fidelity  of  the  simulation  intact,  yet  still  keep 
the  Ethernet  traffic  to  a  minimum  to  reduce  collisions 
and  deferred  frames.  The  fifty  percent  spare  time 
restriction  on  the  trainer  loads  also  makes  it 


data  to  each  other  by  means  of  a  static  memory 
partition,  Datapool.  To  allow  the  separate  trainer 
loads  to  communicate  with  each  other,  each  trainer 
has  its  own  designated  Ethernet  buffers  within  every 
trainer’s  Datapool.  The  Ethernet  buffer  configuration 
is  the  same  in  each  trainer’s  Datapool  and  is 
illustrated  in  Figure  2.  Each  trainer  can  only  write 
into  its  specific  Ethernet  buffer  as  the  other  buffers 
are  continually  overwritten  by  the  Ethernet  transfers 
from  the  other  trainers. 


imperative  to  keep  the  Ethernet  I/O  processing 


addresses  effectively  crrate  a  subgroup  of  nod^  for 
Mch  training  mode.  With  this  implementaion,  data 
is  only  s^t  to  the  trainers  r^juiring  it  for  their 
current  mode,  and  only  one  transmission  is  ne^i^  to 
send  the  same  information  to  several  trainers. 

For  most  software  modules,  it  is  inconsequential 
on  which  OFT  or  SOT  they  are  teing  executed.  For 
some  modules,  however,  there  are  hardware 
constraints  which  require  it  to  know  physically,  on 
which  trainer  it  resides.  For  these  modules,  a  trainer 
ID  is  assigned  during  trainer  activation  based  on  the 
local  network  address  found  in  the  Ethernet 
initialization  file. 


FIGURE  2.  Datapool  Layout 

In  order  for  software  modules  to  be  identical  on 
both  OFTs  and  both  SOTs,  yet  keep  the  physical 
layout  of  the  Ethernet  buffers  the  same  in  all  trainers, 
it  becomes  necessary  for  the  Datapool  dictionaries  to 
be  different  on  the  two  WST  trainers.  For  example, 
the  ownship  aircraft’s  altitude  mnemonic  is  FBALT 
and  the  other  aircraft’s  altitude  mnemonic  is  called 
FBALTB.  On  OFT  #1,  FBALT  will  reside  in  OFT 
ffVs  Ethernet  buffer  and  FBALTB  will  reside  in  the 
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Ethernet  buffer  of  OFT  #2.  On  OFT  #2,  FBALT 
will  reside  in  OFT  WTs  Ethernet  buffer,  while 
FBALTB  will  reside  in  the  OFT  #\  Ethernet  buffer. 
As  the  Datapool  source  files  must  be  the  same,  this 
organization  of  Ethernet  buffers  is  accomplished 
when  the  Datapool  dictionaries  are  built  and  again  is 
determined  by  the  local  network  address  in  the 
Ethernet  initialization  file. 

To  direct  instructor  control  to  each  WST 
selectively  from  a  single  WST  command  station,  a 
Helo  A/B  concept  has  been  implemented.  The 
trainer  from  which  the  scenario  is  being  commanded 
is  denoted  as  Helo  A.  The  other  WST  is  Helo  B.  A 
’Helo  A/B  Source  Display*  switch  designates  to 
which  trainer  the  inputs  will  be  directed  and  which 
WST  values  are  being  displayed. 

Ethernet  Protocol 

A  detailed  description  of  the  Ethernet  protocol 
developed  for  this  system  can  be  found  in  reference 
[4],  with  only  a  basic  overview  of  the  system  being 
provided  here. 

The  Ethernet  transfer  packets  are  data  chained 
Input/Output  Command  Lists  (lOCL’s).  Each  lOCL 
is  six  words  in  length  and  formulated  as  follows.  The 
first  two  words  contain  a  command  to  write  a  header 
which  contains  the  multicast  address,  designating 
where  the  rest  of  the  packet  is  to  be  sent.  The  next 
two  words  contain  a  command  to  write  eight  bytes  of 
4iif  from  a  local  buffer.  The  last  two  words  contain 
a  command  to  write  the  number  of  bytes  (X)  from 
this  trainer’s  Ethernet  buffer  in  Datapool  (BUFFER 
A).  The  local  buffer  specified  in  the  second  two 
words  of  the  lOCL  contains  a  read  command  to  rad 
X  number  of  bytes  into  BUFFER  A.  By  ensuring  the 
physical  address  of  Datapool  is  the  same  on  all 
trainers,  the  receiving  task  then  becomes  a  generic 
receiver  for  all  transmissions  from  every  trainer. 

The  Ethernet  receiving  task  omtinuosly  polls  the 
Ethernet  for  incoming  data  by  executing  a  six  word 
data  chained,  wait  I/O,  lOCL.  The  lOCL  is 
comprised  as  follows.  The  iiist  two  words  contain 
the  read  command  to  read  in  the  Ethernet  header. 
The  next  two  words  contain  the  command  to  read 
eight  bytes  of  data  into  the  last  two  words  of  the 
lOCL,  thereby  making  the  last  two  words  of  the 
lOCL  contain  the  read  command  generated  by  the 
transmitting  trainer.  This  design  greatly  reduces  any 
overhead  that  would  otherwise  be  required  to 
decipher  Bom  which  trainer  the  data  came  in  order  to 
determine  into  what  buffer  to  download  the  data. 

The  lOCLs  executed  by  the  transmitting  module 


are  initialized  during  trainer  activation.  Separate 
packets  are  constructed  to  specify  the  different 
multicast  addresses  for  the  available  modes,  as  well 
as  different  buffer  lengths,  and  the  various  buffer 
addresses.  This  enables  the  transmitting  module  to 
use  the  trainer  mode,  the  trainer  ID,  and  the  current 
frame  to  execute  the  appropriate  lOCL  in  order  to 
direct  the  correct  amount  of  data  to  the  proper 
network  address  at  the  necessary  rate  while  incurring 
the  least  amount  of  processing  necessary  on  both  the 
sending  and  receiving  trainers. 

As  an  example,  the  mode  control  information 
(trainer  state,  operating  coode,  and  integrated  mode 
change  requests)  must  be  transferred  to  all  trainers  in 
all  nKxies.  Also,  it  is  more  effecient  for  a  trainer  to 
transfer  a  complete  buffer  of  data  once  rather  than 
execute  two  transfers  of  varying  lengths  of  the  same 
buffer.  In  MST  nKxle,  some  OFT  data  must  be 
transferred  at  30  Hz.  By  executing  an  lOCL 
specifying  the  broadcast  address  (all  five  trainers)  on 
odd  30  HZ  frames,  and  executing  another  lOCL 
specifying  the  MST  multicast  address  (OFTs  and 
SOTs  only)  on  even  30  HZ  frames,  the  mode  control 
data  going  to  the  AT  is  still  only  sent  at  IS  HZ  while 
the  MST  data  is  transferred  at  30  HZ  and  only  one 
transfer  is  required  from  the  OFT.  The  resulting 
rates  and  denitrations  of  buffer  transfers  for  all 
trainers  range  from  the  sirrqrlcst  and  least  amount 
occuring  when  all  trainers  are  in  indqrendent  mode 
shown  in  Table  2  to  the  most  complex  and  heaviest 
traffic  of  MST  mode  as  outlined  in  Table  3. 


Transfer 

DESTINATION 

Size 

(bytes) 

OFT  1 

OFT  2 

SOT  1 

SOT  2 

AT 

80 

15Hz 

15Hz 

15Hz 

15Hz 

160 

15Hz 

15Hz 

15Hz 

15Hz 

15Hz 

15Hz 

15Hz 

15Hz 

160 

15Hz 

15Hz 

15Hz 

15Hz 

64 

15Hz 

15Hz 

15Hz 

15Hz 

TABLE  2.  Independent  Mode  Buffer  Transfers 
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Trainer 

Source 

Transfer 

Size 

(bytes) 

DESTIWATIOM 

OFT  1 

OFT  2 

SOT  1 

SOT  2 

AT 

OFT  1 

1440 

30Hz 

30Hz 

30Hz 

15Hz 

SOT  1 

1440 

15Hz 

15Hz 

15Hz 

15Hz 

1440 

5Hz 

5Hz 

SHz 

1440 

5Hz 

5Hz 

SHz 

1440 

5Hz 

5Hz 

SHz 

OFT  2 

1440 

30Hz 

30Hz 

30Hz 

15Hz 

1440 

10Hz 

1440 

10Hz 

SOT  2 

1440 

15Hz 

15Hz 

15HZ 

15Hz 

1440 

5Hz 

SHz 

SHz 

1U0 

5Hz 

5Hz 

SHz 

1440 

5Hz 

SHz 

SHz 

1440 

10Hz 

1440 

10Hz 

AT 

64 

15Hz 

15Hz 

15Hz 

15Hz 

TABLE  3.  MST  Mode  Buffer  Tnsmfers 
(WST  #1  In  Commsmd) 


InstTOcJor  Station  Protocol 

The  instractoir  station  receives  input  from  three 
devices;  the  keyboard,  the  trackball,  and  the  digital 
inputs  (DIs)  on  the  instractor  panel.  The  instructor 
station  CRT  displays  are  drivCTi  by  CALCOMP 
VistagiapMc  45TO  display  processors  (one  per 
trainer).  The  display  pages  at©  pre-processed 
graphics  code  subroutine  and  are  table  driven  for 
parameter  updates  and  inputs,  Th^s  tabl^  contain 
information  for  every  dynamic  display  entry  on  the 
page  in  order  to  determine  what  action  to  take  when 
the  item  is  selected,  and  how  to  fom^  the  display  of 
the  item’s  value.  Display  iq>date  is  achieved  by 
building  a  buffer  of  graphics  code  generated  in 
accordance  with  th^e  variable  tables  which,  when 
downloaded  to  the  display  processor,  overlays  the 
current  page,  effectively  updating  the  parameters 
displayed.  Input  for  select^  items  is  achieved  by 
storing  the  new  values  into  the  variable  addresses 
specified  in  the  tabte  for  those  items. 

Although  only  on®  WST  instractor  station  is  used 
in  the  MST  mode,  the  instractor  control  software  still 
runs  in  both  trainers.  The  software  modules 
providing  the  interface  to  the  Helo-B  mstructor 
station  hardware  ignore  any  input  from  those  devices. 
When  the  input  from  the  command  station  is  to  be 
used  by  both  trainers,  it  is  processed  as  normal  by 
the  Helo-A  instractor  control  software,  but  in 
addition,  this  software  sends  flags  and  control 
variables  to  the  Helo-B  station.  If  the  input  is 
specific  to  one  trainer,  the  Helo  A/B  designator 


determines  where  the  input  is  directed.  If  Helo-A  is 
selected,  the  instructor  control  software  in  Helo-B 
does  nothing,  while  the  Helo-A  software  processes 
the  input  the  same  as  in  independent  mode.  When 
the  "Helo- A/B  Source  Display"  switch  indicates 
Helo-B  is  selected  the  Helo-A  instractor  station 
interface  modules  seed  the  input  over  the  Ethernet, 
where  the  Helo-B  modules  process  it  as  though  it 
were  generated  by  the  Helo-B  mstructor  devices. 

The  CRT  displays  for  the  Helo-B  trainer  are 
generated  and  updated  in  MST  mode  in  the  same 
manner  as  they  are  in  any  other  mode.  However,  on 
the  command  station,  when  the  "Helo-A/B  Source 
Display"  switch  indicates  Helo-B  is  selected,  the 
CRTs  must  reflect  the  HELO-B  values  for  the 
parameters  displayed. 

To  preserve  the  table  driven  page  design  without 
having  to  add  additional  page  variables  for  the  Helo- 
B  display,  the  stipulation  that  the  trainer  software  be 
identical  for  both  trainers  was  put  to  use.  Controls 
were  added  to  ensure  both  OFT  trainers  (as  well  as 
both  SOTs)  display  the  same  CRT  pages  in  MST 
mode.  In  this  way,  the  item  select  information  and 
data  input,  when  sent  over  the  Ethernet  to  Helo-B, 
can  be  processed  by  the  page  input  module  and 
loaded  directly  into  the  selected  item’s  variable 
addresses  as  though  it  was  entered  from  the  Helo-B 
keyboard. 

The  restriction  of  guaranteeing  both  trainer  CRTs 
display  the  same  page  in  MST  mode  is  also  used  for 
the  display  of  Helo-B  mfonnation.  By  adding 
handshaking  controls  between  the  Ethernet 
transmitting  module  and  the  display  update  routine, 
when  Helo-B  is  sel«;ted,  its  graphic  update  buffer 
can  be  sent  to  the  Helo-A  page  update  module  and 
downloaded  to  the  Helo-A  graphics  processor, 
thereby  redrawing  the  page  parameters  to  reflect 
Helo-B  values. 


SUMMARY 


The  incrrased  Ethernet  traffic  in  the  MST  mode 
does  heighten  the  number  of  deferr^  transfers  and 
collisions.  However,  the  only  aira  where  it  proved 
critical  was  in  the  instractor  control  software. 
Fortunately,  software  measures  could  be  implemented 
to  protect  the  simulation  in  the  areas  where  such 
occurences  would  be  fatal.  With  the  Ethernet  protocol 
developed,  the  highest  rate  of  transfers  needed  for  the 
MST  mode  were  permitted  without  degrading  the 
simulation.  TTie  added  feature  of  being  able  to  use 
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one  command  station  to  control  the  entire  scenario 
yet  still  control  the  WSTs  individually  results  in  a 
reduced  instructor  workload  for  a  highly  complex 
training  scenario  while  still  having  full  monitoring 
capabilities  for  both  trainers.  Through  these  technical 
modeling  implementations,  highly  diversified 
capabilites  have  been  made  available  in  a  single 
training  system.  The  MST  mode  enhancements  in 
the  areas  of  communications,  formation  flight,  and 
acoustic  effects  allow  for  coordinated  ASW  crew 
training  in  a  common  tactical  problem  world  and  still 
provide  the  high  fidelity  of  simulation  available  in  an 
independent  mode. 
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Matter  Hiysics.  Simulation  programs  within 
Reflectone  have  been  primarily  U.S.  Navy  ASW 
Helicopter  Weapon  System  Trainers  including  the 
SH-2F.  SH-3H,  and  SH-60F.  Experience  at 
Reflectone  has  been  in  the  areas  of  Tactical  Systems 
Engineering  and  Program  Engineering  leading  to  the 
current  position  of  Director  of  Program  Engineering, 
Military  Programs. 
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ABSTRACT 

The  dramatic  increase  in  aircraft  complexity,  the  high  cost 
per  hour  to  fly  these  complex  aircraft,  and  continued 
resistance  to  low  altitude  military  training  require  a  major 
change  in  the  development  of  aircrew  training  devices.  No 
longer  can  the  simulator  serve  as  a  procedural  trainer;  it 
must  do  much  more.  The  question  is  how  best  to  procure  a 
trainer  that  does  so. 

Recently,  there  has  been  much  written  on  the  acquisition  of 
weapon  system  trainers  through  the  prime  weapon  system 
contractor.  Aside  from  the  increase  in  cost,  this  approach 
can  preclude  the  acquisition  element  and  the  user  from 
making  direct  inputs  into  the  training  system's  development. 
This  can  result  in  the  delivery  of  a  very  expensive, 
capability  limited  training  system  to  the  user.  The 
associate  contractor  arrangement  that  CAE-Link  has  with 
Northrop  on  the  B-2  program,  combined  with  some  innovative 
training  development  concepts,  made  the  B-2  Aircrew  Training 
Device  (ATD)  procurement  the  model  for  future  training 
system  acquisition.  We  in  the  B-2  program  are  in  the 
process  of  building  not  only  the  most  complex  aircrew 
trainer  yet,  but  at  the  same  time,  the  best  device  for 
training  crews .  The  complexity  of  the  B-2  Bomber  requires 
nothing  less. 


INTRODUCTION 

The  development  of  the  B-2  Bomber  presented  both  unique 
challenges  and  opportunities  for  the  concurrent  development 
of  its  ATDs.  There  are  several  theories  as  to  the  "best" 
strategy  for  simulator  procurement,  including  acquisition 
through  the  prime,  the  newest  approach  to  training  system 
acquisition.  The  B-2  acquisition  was  the  successful 
application  of  a  traditional  strategy  aided  by  the  ability 
to  be  more  innovative  because  of  the  program  environment. 
CAE-Link,  Link  Flight  Simulation  Division,  won  a  separate 
competition  to  develop  the  ATDs.  Aircraft  data  were 
provided  to  CAE-Link  through  an  associate  contractor 
agreement  (ACA)  with  Northrop,  the  B-2  prime.  Northrop  also 


is  responsible  for  developing  of  the  entire  B-2  training 
program^  to  include  all  simulator  courseware,  primarily 
mission  scenarios =  The  tie  between  Northrop  and  Link  is 
important,  for  it  is  one  of  the  keys  to  success  of  the  B-2 
ATDs»  Although  the  Link  contract  is  a  government 
procurement,  Northrop  is  fully  responsible  for  the  success 
of  the  B-2  training  program,  of  which  the  ATD  is  the 
principal  piece  of  hardware  for  aircrew  training.  This 
drives  a  close  working  relationship  between  the  contractors, 
contributing  to  the  government's  ability  to  provide  a 
quality  training  system. 

There  were  three  major  programmatic  challenges  to  the  B-2 
ATD  acquisition  to  be  overcome.  First  and  foremost,  the 
user,  the  Strategic  Air  Command,  required  an  operational  ATD 
be  in  place  and  ready  for  training  (RFT)  prior  to  delivery 
of  the  first  operational  aircraft  to  Whiteman  AFB,  MO,  The 
ATD  had  to  identically  match  the  delivered  aircraft,  the 
second  major  challenge.  The  third  challenge  was  developing 
the  ATD  at  the  same  time  as  the  aircraft.  Handling  the 
massive  influx  of  changes  as  the  aircraft  developed  required 
some  very  innovative  techniques  in  engineering,  contracts, 
and  management , 

Two  independent  factors  combined  to  make  the  meeting  of 
these  challenges  a  success  story  in  the  arena  of  concurrent 
simulator  development.  The  B-2  Program  Management  Directive 
(PMD)  called  for  a  streamlined  management  approach  to  the 
entire  B-2  acquisition.  The  high  level  of  security  at  the 
start  of  the  program  kept  the  B-2  team  small.  For  example, 
the  B-2  Air. Staff  representatives  reported  directly  to  the 
Secretary  of  the  Air  Force,  Streamlined  management  drove 
flexibility,  while  security  drove  small  team  size,  a  great 
combination  for  innovative  acquisition. 

It  is  important  to  note  that  during  the  building  and 
execution  of  the  acquisition  strategy,  all  acquisition 
guidelines  and  regulations  were  followed  to  the  letter.  Our 
advantage  was  that  our  decisions  were  reviewed  by  higher 
levels  of  management  that  were  new  to  training  acquisition. 
They  trusted  the  judgement  of  the  ATD  team,  and  that  trust 
allowed  the  implementation  of  many  of  our  innovations.  The 
small  acquisition  team  also  aided  in  that  many  of  the 
upper-level  decisions  were  delegated  to  the  B-2  core  team, 
dramatically  reducing  the  number  of  individuals  involved  in 
the  decision  making  process. 

One  other  factor  contributed  to  the  innovations  that  we  were 
able  to  use.  Because  cutting-edge  technologies  such  as 
those  used  to  develop  the  B-2  ATD  pose  greater  risk,  the 
Full  Scale  Development  (FSD)  portion  of  the  contract  was  a 
cost  plus  incentive  fee  with  award  fee  arrangement.  This  is 
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.  controversial  approach  the  Jo^most  objection^being  that 

it  virtually  guarantees  i^ed  degree  in  the  B-2 

program.  This  has  i  Question  is  whether  the 

ATD  acquisition.  The  is  worth  the  value  to 

predicted  and  The  importance  of  the  simulator 

the  B-2  program  as  ® *  ^Pj^^thronly  answer.  The  high 

^°a^''^i^-tL^;rir-  Pius  :uaUy^co|perative^^^^^^ 

environment  made  some  of  our  in  work  on  a  more  common 

S?iTixei  rricH?lpra?«ngem:nt.  Other  programs  can  take 
ad“n?igfofiu  of  the  things  the  B-2  ATD  has  done. 

INNOVATIONS: 

Managed  Through  the  Aircraft  SPO 

classified  nature  of  the  program  allowed 
As  mentioned,  the  .  simulator  acquisition, 

many  avenues  to  be  explored  in  transferred  to  more 

HowLer,  many  “I ""e  innovations  can  be  transferr^^^^^^^ 

oir-iitfk-iiist^ice^favl  ul  was^management  under  the 
aSspices  of  the  B-2  System  PJ°9ua™  in  ?he  Aii  Force 

majority  of  the  training  -gjjj  gpo.  The  movement  of 

are  bought  through  the  Twining  System  SPO^  ^ 

^crosrr^«kirg‘’rriitrnsrip^w!tS^the  aircraft  developers. 

With  concurrent  development  of  the  aircraft  and^tte  ATD 

being  the  single  greates  for^better  communications  in 

advantage  of  "f  ®  l^er?  aUcrIlt  engineering 

frSnr^ThaMJpa  t  was  thoroughly  briefed  « 
rnflii:rirthr£u5°geting  for  the^overall^EC^^ 

improved  engineering  cornu  The  first  ATD  was 

dividend  to  the  ^aircraft  delivered  to  SAC, 

designed  to  match  the  fir  •  aircraft  development,  the 

SAC-1.  AS  is  normal  nex?  airLaft  off  the 

engineering  community  1°°^®^  ®  maioritv  of  their  effort  on 
production  line  and  ‘-"-d  ^OpmSS  CwLO  SAC-1  worked  as  a 
that  aircraft.  „  to  eLure  that  short-term 

backgpund  sanity  check  t  g_2  aircraft  in  the 

solutions  ^^^/‘^geJeral  potential  problems  have  been 

long  run.  Already,  ^  pxamole,  the  original 

averted  due  to  ."'®‘:^CU^  computer  complex  on  the  aircraft 

avionics  control  unit  (^  )  ^Q^throp's  original  solution 

required  some  vehicle  and  saved  money,  but  the 

IO0a0t''?rtri?D0  was  considerable  and  act^l^ 

IZ  ?0f"t0l  OOOlaOLfOlOnr^rh-thfi^O  cLe  up  with  a 
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solution  that  fixed  the  problem  and  saved  money  on  both 
programs « 

Another  major  advantage  has  been  the  ease  of  access  to  SPO 
engineers  working  the  aircraft  subsystems  =  From  the 
beginning^  these  engineers  were  educated  on  the  team 
approach  to  the  B-2  development  and  how  critical  the  success 
of  the  training  program  was  to  the  success  of  the  entire 
program,,  With  this  in  mind,  there  has  been  little 
difficulty  in  gaining  participation  from  engineers  to 
support  ATD  development,  up  to  and  including  trips  to  the 
Link  facilities  to  better  explain  subsystem  operations  to 
Link  engineers.  Both  SPO  and  Link  engineers  benefitted 
greatly  from  this  interface. 

The  final  major  advantage  of  being  under  the  wing  of  the 
aircraft  SPO  is  funding.  Being  funded  as  a  weapons  system 
package  with  the  ATDs  being  part  of  the  entire  system  was  a 
great  advantage.  As  will  be  explained  later,  maintenance  of 
concurrency  is  expensive.  The  parallel  development  of  the 
ATD  with  the  aircraft  in  the  FSD  portion  of  the  program 
created  the  potential  for  some  very  large 
"Unknown-Unknowns, "  The  traditional  simulator  funding 
profile  would  not  be  able  to  support  the  management  reserve 
required  to  adequately  cover  the  program  in  this  area 
because  of  the  large  number  of  changes  in  the  aircraft. 

With  the  ATD  being  covered  by  the  management  reserve  of  the 
aircraft  budget  these  problems  were  overcome.  The  cost  of 
the  ATD  program  is  a  very  small  percentage  of  the  total  B-2 
program  costs,  and  this  allows  for  some  flexibility  in 
financing, 

iraOVATIOMS 

The  Link-Worthrop  Connection 

Another  major  plus  for  the  success  of  the  B-2  ATD  was  the 
excellent  relationship  that  developed  between  the  prime 
aircraft  contractor,  Northrop,  and  Link's  Flight  Simulation 
Division.  Initially,  Northrop  established  a  boilerplate  ACA 
with  all  the  contractors  bidding  on  the  B-2  ATD  project. 
After  contract  award.  Link  tailored  this  arrangement  to 
maintain  the  flow  of  data  from  Northrop,  Establistoent  of 
an  ACA  was  not  an  easy  task.  Working  out  the  details  for  a 
smooth  flow  of  aircraft  data  to  Link  took  a  great  deal  of 
time  and  effort.  This  ACA  continues  to  change  as 
requirements  evolve.  Currently  an  amendment  to  the  ACA  is 
in  progress  to  allow  Northrop  instructors  to  be  trained  in 
ATD  operation.  This  type  of  flexibility  in  the  ACA  is  a 
requirement  for  success.  As  a  result  of  this  ACA,  a  solid 
commitment  from  Northrop  was  realized  with  the  establishment 
of  a  group  of  five  employees  whose  sole  purpose  is  to 
provide  and  track  the  data  Link  needs  to  build  their 
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simulator.  In  addition  to  this  group,  four 

are  co-located  with  the  Northrop  team  to  ,  s 

1  ont i on  interoretation,  and  transfer.  The  Link  group  s 
InitHl  wafto  idaitify  to  Northrop  the  Informatron 

reguired  by  the  simulator  engineers. 

As  the  aircraft  matured,  the  members  of  this  became 

involve^in  the  earliest  phases  of  the  aircraft  change 
process.  Link  now  receives  preliminary  word 
changes  very  early  in  the  Northrop  change 
alone  has  saved  the  program  large 

postponing  work  that  was  to  change.  team 

resDonsibility  for  total  training,  also  uses  this  ATD  team 
to  better  integrate  the  simulator  with  the  rest  of  the 

training  system.  The  positive  working  B  2 

g^Si^hL  Len  a  major  contributor  to  the  success  of  the  B-2 

ATD  development  effort. 

INNOVATIONS 

Concurrency 

Since  the  introduction  of  simulators  modeled  after  specific 
aircraft,  the  greatest  challenge  to  the  ^ 

has  been  to  accurately  mirror  the  aircraft  in  a  timely 
fashion.  The  B-2  ATD  has  had  the  commitment  of  the  SPO 
Director  that  the  delivered  ATD  will  be  the  s^e  as  the 
delivered  aircraft.  This  solid  support  from  the  front 

ofkce  allowed  us  the  opportunity  to  d°  ^  ijr^qiificant 
things  from  a  contractual 

(ClaL  One)  changes  to  the  aircraft  that  affect  the 
si^lator  Automatically  become  Class  One  changes  to  the 

<?imulator  at  the  same  time.  The  reason  for  this  is  tnar  u 
SPO  Management  has  signed  up  to  aircraft 

ECPs  to  make  Class  One  changes  to  the  simulator. 

we  also  developed  a  novel  method  for  handling  smaller  (Class 
?wo)  changes.  In  traditional  procurement  management,  small 
changes  are  accumulated  over  a  period  of  time  and 
incoroorated  into  the  simulator  through  block  updates.  Wit 
our  commitment  to  concurrency  and  the  heads  up  we  J^eceive 
i-Vie»  amn  team  at  Northrop  on  aircraft  changes,  we 

Apprnval  was  granted  by  the  ATD  Pr°9>^f 
,  _  CBO  amn  team  for  all  but  one  of  the  submitted 

changes  This  process  was  in  place  through  system  Critical 
Sn  Revlei  (CDR)  and  accounted  for  109  changes  to  the 
simulator  of  which  only  seventeen  were  above  the  $25,000 
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threshold o  It  helped  keep  the  ATD  concurrent  with  the 
aircraft  and  prevented  millions  of  dollars  being  spent  on 
ECP  preparation  and  rework  for  changes  that  would  have  been 
automatically  approved  anyway „  After  the  period  of 
performance,  the  cost  for  these  changes  was  submitted  ^he 
government,  which  had  been  tracking  and  budgeting  for  these 
changes,  A  similar  program  has  been  developed  to  manage 
change  after  system  CDR, 

In  addition  to  managing  real  time  changes,  an  engineering 
effort  was  put  forth  to  handle  future  concurrency  issues. 

The  B"2  bomber  is  an  incredibly  software  intensive  weapon 
system.  Because  of  the  size  of  the  software  effort  it  was 
obvious  from  the  beginning  that  there  would  be  large  nu^ers 
of  software  changes.  Again,  as  with  hardware  changes,  the 
current  philosophy  is  to  hold  these  changes  until  a  block 
update  is  economically  possible.  We  have 
problem  by  designing  the  ATD  to  run  on  the  same  software 
that  is  used  in  the  aircraft.  By  doing  this,  ^software 
change  to  the  aircraft  can  simultaneously  be  ^^de  to  the 
simulator  with  identical  results.  Our  only  shortfall  in 
this  area  is  that  we  are  entirely  dependent  on  aircraft 
software  development  in  order  to  allow  the  ATD  to  fly  like 
the  aircraft,  and  this  has  caused  some  perturbations  in  the 
ATD  schedule  during  aircraft  development.  This  drawback  a 
recognized  in  the  original  acquisition  ® J 
decision  was  made  to  accept  simulator  schedule  impacts  to 
allow  for  a  concurrent  machine.  Additionally,  this  was  one 
of  the  drivers  to  the  cost  plus  contract. 


INMOVATIOMS 

Operations  and  Training  Analysis  Group 
( OTAG ) 


The  B-2  Bomber  is  an  extremely  complex  weapon  system.  Early 
in  its  development  it  became  apparent  that  the  training 
device  had  to  offer  a  degree  of  fidelity  and  training  _ 
capability  that  was  beyond  any  device  yet  developed,  T 
new  groups  were  developed  to  make  this  happen, 

Operations  and  Training  Analysis  Group  (OTAG)  and  the 
Training  Systems  Working  Group  (TSWG) .  OTAG  is  an 
.  organization  of  Link  employees  who  are  former  military 

aircrew  instructors.  This  group  is  represented  at  all  ma^or 
company  meetings,  both  engineering  and  management,  ^nd  is 
crucial  in  assessing  the  training  impact  of  all  changes  to 
the  simulator.  In  addition  to  assessing  training  impacts  of 
engineering  changes,  OTAG  members  bridge  the  9®? 
simulator  specifications  and  training  needs.  They  do  this 
by  serving  as  a  kind  of  surrogate  user,  putting  themselves 
in  the  position  of  both  student  and  instructor.  However, 
ihey  are  not  an  independent  team;  their  objective  is  to  put 
theLelves  in  the  place  of  the  user.  In  this  capacity,  they 
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also  assist  engineering  cLtractor  mission  testing 

during  developmental  testing  and  to  replicate 

(to  be  "|*^ler  job  thL  building  a  ei^bletor  that 

the  aircraft  IS  a  s^pier  1  training  program.  O^AG  s 

gr^tSsf  res^nribfli?y  is  t^  maintain  the  "training  first 

baseline  of  the  program. 

Sn^systems  Wording  Group  (TSWG, 

While  the  engineering  Ptocess  yields  a^|i»ulator.^by^itself 

it  does  not  yield  ^  simulator  becomes  an  even 

to  ensure  that  a  well  d  g  |  the  high  degree  of 

better  draining  device^  T  in  the  beginning 

training  was  met,  the  ^  ^  ^his  team  was  composed  of 

stages  of  the  ATD's  development.  These  four 

members  from  SAC,  the  SPO,  ^  all  four  services  and 

groups  contributed  of  more  than  100,000  hours 

lommLcial  airlines,  h  *  “Ltructor  time  ^om 

of  flying  time,  half  of  wni  capability  of  the  B-2 

this  core  of  of  their  work  was  accomplished 

ATD  was  formed.  The  °oud  can  be  called  at  any  time 

prior  to  system  9°*^' ®  ^^°eL?inue  to  surface.  For 
to  resolve  training  issu  instructor  panel  were 

example,  ell  of  PJ^^^Th^work  done  by  the  TSWG  revolved 

finalized  prior  to  CDR.  jne  wo  ^nd  training 

around  two  magor  training  benefits  were  weighed 

performance.  To  give  an  example,  the  idea  of 

against  program  -(gnt  seat  trainer  his  own 

giving  the  instructor  _,f.ed  After  discussing  the  low 

Jadar  tracking  "^ndle  was  raise^  At  ,,o„,,apt 

utility  of  this  in  the  FBlll  Bo^  the 

was  rejected.  On  the  positive  sra,  group. 

s“te-of-the-art  i^tructor  console^c^e^ou^  acenario  to  be 
This  PdAdl^til  allow  g  instructor  can  spend  far 

made  quickly  .  ®^J®®^tor  Ind  less  time  as  a  simulator 
more  time  as  an  ^-^^tructor  a  innovative  groups  used 

operator.  The  the  B-2  ATD  from  becoming  D^st 

rnofhti  =0^  gMA 

Iong’'wly°ireiti??ng  this  philosophy  was  retained. 

INNOVATIONS  a.  • 

Operational-Based  Testing 

The  importance  of  training  evenjeaches^he^test^^e^ 

b-2  ATD  acquisition.  Tradir  ^  item  in  the 

testing  tLing  months.  Often  results  of 

orUttin^ware  strLned  relationships  among  user. 
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SPO^  and  contractor^  a  very  large  stack  of  test 
discrepancies^  and  a  simulator  in  which  training  was 
perceived  to  be  a  secondary  function. 

The  B“2  ATD  test  approach  de-emphasizes  the  importance  of 
specification  testing.  Instead  of  the  traditional  test^  an 
operational  testing  program  was  developed,  its  basic 
premises  test  the  capability  to  train.  There  is  a 
developmental  test  and  evaluation  (DT&E)  period  of  about 
four  weeks,  during  which  stand-alone  systems  and  subsystems 
are  thoroughly  tested  (to  specification  requirements). 

These  systems  include  things  such  as  the  motion  base,  image 
generation  system,  and  digital  radar  landmass  simulation 
(DRLMS),  After  successful  completion  of  DT&E,  operational 
test  and  evaluation  (OT&E)  is  started.  The  TSWG 
participants  identified  what  needed  to  be  trained  in  the 
simulator  both  from  an  initial  instruction  and  a 
continuation  training  perspective.  The  operational  test 
plan  was  built  around  these  requirements  for  training.  Each 
training  function  would  be  accomplished  in  the  simulators 
how  it  performed  the  function  as  compared  to  a  predetermined 
threshold  determined  the  success  of  the  test.  To  make  this 
a  smooth  operation,  a  copy  of  the  operational  test  plan  is 
given  to  the  contractor,  who  runs  all  the  tests  prior  to 
calling  in  the  Air  Force  to  run  the  government  testing.  The 
contractor's  mission  test  is  performed  by  OTAG,  If  this 
group  does  a  thorough  job,  the  government  OT&E  will  be 
complete  in  the  five  weeks  scheduled.  Although  important, 
this  type  of  testing  is  not  conducive  to  the  total  training 
solution. 

Even  test  discrepancies  (TDs)  are  based  on  training.  All 
TDs  are  written  and  prioritized  by  their  impact  on  training. 
The  priority  of  these  discrepancies  ranges  from  stopping  a 
test  to  items  that  can  wait  for  reassembly  at  Whiteman  AFB, 
However,  the  test  methodology,  with  the  contractor  doing  its 
mission  test,  should  significantly  limit  the  number  of  TDs. 
This  test  approach  is  a  major  departure  from  the  past  that 
can  pay  big  dividends  in  the  future, 

INNOVATIOMS 

Things  That  Didn't  Work 

With  all  the  things  that  were  tried  on  this  program  that 
were  a  success  we  did  have  some  failures.  Holding  onto  the 
belief  that  the  first  delivered  ATD  would  have  total 
aircraft  capability  caused  some  problems ,  We  thought  that 
our  innovations  would  allow  this  to  occur.  If  we  had 
created  a  phased  approach  at  the  beginning,  we  may  have 
saved  some  time  and  money.  We  learned  that,  no  matter  how 
creative  you  may  be,  there  is  always  gravity. 
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CONCLUSIONS 


Much  innovative  thought  went  into  the  development  of  the  B-2 
ATD.  This  was  a  necessity.  The  challenges  of  the  program 
required  a  different  approach  to  simulator  acquisition  in 
order  to  get  the  job  done.  Have  we  been  successful?  As  of 
this  writing  the  schedule  shows  a  delivered  ATD  with  the 
same  training  capability  as  the  aircraft,  prior  to  that 
aircraft's  arrival.  Not  everything  we  tried  will  work  on 
all  trainer  acquisitions,  but  we  hope  that  this  paper  will 
stimulate  thought  on  alternative  acquisition  strategies  for 
other  systems . 
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abstract 

"Learning  by  doing”,  often  by  children  ifarn^from  their  parents,  by 

transferring  knowledge,  and  it  is  stiH  on  actions  and  by  receiving  constructive 

experimenting  and  attempting  training  systems  d'^^ 

feedback  on  their  performance.  Early  co  p  ^^3  transfer  of  training  benefit  that 

to  practice  behaviors,  and  oonsequ  J  .  .  provide  the  opportunity  for  unrestricted 

experiential  learning  affords.  prohibits  their  widespread  use 

time  on  small,  portable,  inexpensive  computers. 

The  Fuel  System  Simulator  is  a  performance 

combination  of  tutorials,  guided  .  ^ce  training  problem.  With  a  design  that 

assessment  to  solve  a  app^  encourages  heuristic  learning,  resulting 

introduction 

The  simple  experiences  of  remain  with 

Memories  of  burned  fingers  ;;"®J;'®y''®of  our  weVmeaning  parents.  The  things  we 

us  far  longer  than  the  repeated  ^dmonit  js  o  pa^e^ful 

learned  long  ago,  many  of  and  then  saw  the  immediate  consequences 

Toractr  rtraining  today  as  they  were  then. 

Historically,  j®e  to  survive, 

from  their  parents,  often  by  ®  °  edge  and  exp^^^^  Children  attempted  to  • 

Parents  were  the  models  who  shar  feedback  and  corrective  advice.  Their  learning 

duplicate  their  parents'  actions,  and  9  ent  of  psychomotor  skills.  This  was  one 

was  based  on  repetition,  practic  knowledge  and  it's  still  one  of  the  best.  As  humanity 
of  the  earliest  methods  of  t^a'^sferrmg  k  ^,1  ^as  to  know, 

became  more  organized  and  sop  i  '  teachers  and  stand-up  training. 

Consequently,  specialization  occu  •  -ofr^plicated.  People  had  to  be  taught  how  to  use 
As  societies  developed,  life  grew  languages  and  how  to  do  things  better  and  faster. 

and  training  increased  exponentially. 
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Eventually,  analog  and  digital  computers  appeared  on  the  scene.  Their  invention  meant 
that  procedural  training  could  be  supported  by  automated  training  devices;  that  a  computer 
would  do  some  of  the  functions  normally  performed  by  a  teacher.  The  result  was  the 
emergence  of  a  whole  new  discipline  -  -  computer-based  training  (CBT).  With  the  advent  of 
computers  and  CBT,  technology-based  training  enhancements  could  also  be  explored. 
Authoring  systems  were  developed  to  support  uniformity  and  increase  student  throughput. 
Colorful  computer  graphics  and  "user  friendly"  input  devices  were  incorporated  to  add  interest 
and  promote  interactivity.  All  of  these  advances  had  benefits,  but  they  also  shared  a  major 
problem.  They  did  not  allow  for  the  "what  if  I  do  this?"  scenario,  i.e.,  unrestrained 
experimentation  and  practice  to  see  what  will  happen.  The  value  of  "learning  by  doing  was 
not  being  captured. 


ENCOURAGING  TRANSFER  OF  LEARNING 

The  incorporation  of  simulation  into  CBT  occurred  because  training  professionals 
realized  that  an  important  learning  component,  the  ability  to  practice  actual  tasks,  was  missing 
from  existing  training  devices.  Computer-based  simulations  were  developed  for  a  wide  variety 
of  sophisticated  systems.  Inevitably,  the  more  complex  the  training  task^  the  more 
complicated  the  training  device,  and  the  simulation,  became.  Fidelity  (i.e.,  how  jea  'Stic  is 
it?)  became  one  of  the  key  goals  for  this  new  medium.  It  was  believed  that  high  fidelity  (and 
consequently  high  cost)  would  ensure  true  learning. 

Each  of  us  involved  in  training  has  wondered  "is  my  training  going  to  make  a 
difference?"  "Will  the  learning  transfer  to  the  real  world?"  "Will  the  trainees  actually  make 
changes  that  will  improve  performance,  or  is  the  training  just  another  break  from  the  daily 
routine?"  Research  has  shown  that  practice  combined  with  feedback  on  performance 
produces  a  very  high  level  of  learning  transfer.’  Training  professionals  currently  working  in 
the  field  agree.  In  a  recent  article  in  Training  magazine.  Dr.  Jack  Asgar,  states. 

"If  you  expect  trainees  to  apply  what  you  are  teaching  them  back  on  the  job,  design 

learning  activities  that  require  them  to  use  the  same  behavior  they  must  use  in  real 

life."^ 


SIMULATION  AS  A  TRAINING  SOLUTION 

Determining  the  best  training  approach  for  a  particular  system  or  piece  of  equipment 
means  evaluating  alternatives  against  a  number  of  goals.  Along  with  the  training  goals,  we 
also  have  to  consider  access  and  availability,  technical  issues,  system  life  expectancy,  safety, 
supportability  and,  of  course,  cost.  As  an  example.  Table  1  illustrates  three  options  for 
training  an  operator  to  use  a  piece  of  equipment  in  a  cockpit,  in  this  case  a  fuel  control  paneL 
The  options  all  offer  a  high  degree  of  functional  fidelity,  but  with  varying  cost,  interface,  and 

revision  considerations. 


’  Bond  Cdr  Jeffrey  P.,  Royal  Navy  Directorate  of  Naval  Education  and  Training  Support. 
"Computer  Based  Training  -  Hit  Or  Myth",  Proceedings  of  the  10th  Interservice/Industiy. 
Training  Sv.stems  Conference,  December  1988. 

"  Asgar,  Jack:  "Give  Me  Relevance  or  Give  Me  Nothing",  Training,  July  1990,  p.  50. 
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Table  1-1.  Tradeoff  Analysis  of  Equipment  Training  Options 


PANEL  OPTIONS 

FIDELITY 

COST 

INTERFACE 

UPDATE 

Actual  Panel 

Tactile  -  High 

Visual  -  High 

Functional  -  High 

High 

Possibly  Complex 

May  Be  T ime 
Consuming 

Emulated  Panel 
(3-D  Mockup) 

Tactile  -  High 

Visual  -  High 

Functional  -  High 

Hediun 

High 

Less  Complex 

May  Be  Time 

Cons  lining 

Interactive  Graphic 
Representation 

Tactile  -  Low 

Visual  -  Moderate 

Functional  -  High 

Lou 

Minimal 

Complexity 

Easi ly  Done 

Unfortunately,  in  aviation,  be  it  military  or  commercial,  we  can  not  often  give  trainees 
the  opportunity  to  practice  in  real  time  on  real  equipment.  It  is  far  too  costly  for  rriany  training 
applications,  and  the  consequences  of  mistakes  can  be  catastrophic.  For  these  high  risk/high 
cost  applications,  simulation  has  become  widely  accepted  as  the  preferred  way  to  train. 
Unfortunately,  even  simulators  have  their  limitations,  as  we  try  to  meet  widespread  training 
needs  with  a  few  very  expensive  resources.  In  the  attempt  to  achieve  ultimate  fidelity,  the 
industry  has  created  trainers  that  are  too  expensive  for  widespread  use.  The  solution  to  this 
dilemma  is  functional  simulation  of  a  complex  system,  run  in  real  time  on  small,  portable, 
inexpensive  computers. 


A  BETTER  WAY  TO  TRAIN 

This  solution  has  proven  to  be  particularly  effective  in  the  training  of  discrete, 
specialized  systems,  i.e.,  when  there  is  a  specific  training  problem.  Within  the  aviation  field 
alone  there  are  a  number  of  specialized  systems  which  must  be  understood  and  mastered. 
Complex  new  navigation  systems,  for  example,  require  the  aircrew  to  follow  correct 
procedures  and  make  appropriate  navigation  decisions.  Maintenance  systems  are  also  well 
suited  to  this  technique. 

The  Fuel  System  Simulator  (FSS)  is  a  maintenance  application  of  this  approach, 
developed  in  response  to  a  real  world  problem.  The  FSS  is  a  powerful,  low  cost  training  tool 
that  goes  "back  to  basics",  using  learning-by-doing  training  concepts,  but  implementing  them 
with  state-of-the  art  simulation  technology.  It  trains  airline  maintenance  personnel  to  perform 
fueling,  defueling,  and  fuel  system  troubleshooting  on  the  DC-8  aircraft.  It  combines  self- 
paced  tutorial  lessons  and  guided  instruction  with  unrestricted,  real-time  practice  -  -  interactive 
freeplay.  With  its  dynamic  fuel  system  simulation  and  interactive  fuel  panel  representations, 
it  essentially  "brings  the  aircraft  equipment  to  the  student".  The  simulation  concept  for  this 
particular  trainer  is  threefold.  Functional  simulations  of  the  Fuel  System  Control  Panel, 
important  Subsystem  Operations,  and  Fuel  System  Schematics  are  combined  and  correlated 
in  a  single  training  system  that  resides  on  a  desktop  PC.  By  concentrating  on  functional 
fidelity,  the  FSS  simulates  those  things  which  are  critical  to  achievement  of  the  training 
goals,  but  in  a  way  that  is  also  affordable. 

The  training  system  is  comprised  of  four  training  modules  and  an  optional  authoring 
module. 


The  Controls  ancJ  Displays  ^  ,he''touchsaee'n'’?tud'er«s^can  "explore"  the 

function  and  operation  of  the  selected  item. 

The  Schereatrcs  Modole  ,s  also  a  “ 

::re«  sch^rtfc  tr:“can  compare  the  schematics  o,  the  two  different 

DC-8  airframes. 

The  Demonstration  3^f,uerplnel'se^^ 

-r  «  -Lcuences  of  their  actions. 

The  Freeplay  Module  gives  ''ainees  °9P°™hdJ  ,1  ,^inc,ional 

exercises  created  by  a  knowledgeable  '"f  or  to  diagnose  and 

fuel  system  simulation  to  Pa'X'mi'f  y  ar^dsJ^ect  components  to  change,  test,  or 
correct  problems.  Trainees  can  '^enti  Y  can 

repair,  and  can  P®''®™  Xob"  but  the  costs  of  an  error,  or  the  practice  itsed, 
operate  iust  as  '.‘'PV  "pXy  Module  also  automatically  tracks  ahd  records 

have  been  eliminated.  The  Freep  v  ^  At  the  end  of  an  exercise, 

student  actions  for  Xrv'oMto  achons  and  evaluate  their  performance  against 
'that  ofan  expert,  mshuctors  can  review  student  performance  at  their  cohvehrence  by 
recalling  the  desired  student  file. 

This  set  of  capabilities  has  of  simulation  with 

or  through  real-life  experience  on  the  jr)  .  ..n  that  students  can  select  what  they  wish 

the  advahtages  of  random  access  to  X  sSe  pSrty  and  low  cost  means 

,0  learn  and  can  progress  at  the.,  o*;'  X  mwSg  productivrty  losses,  strain  or, 

'rare“n;rcik.ies,  and  the  costs  of  ^nging  students  to  7-Xca°:bru:'ed 

a"a’’n  fnteg'tar't  ou  I'a'rg'er'treSg  P^r'am,  or  can  serve  as  a  performance  support  system, 

providing  "just  in  time"  training  on  the  job. 

The  FSS  design  has  several  key  'PP--X;fverrpiab,l*Th\^  apTic^^r " 

and  effectiveness,  '.‘p ''to  otto  maintenanM  functions,  or  to  other  training 
applied  to  other  avionic  ^YSjems  -W  the  design  features  a 

problems.  Since  retention  of  friendly"  tLchscreen  interface  and  extensive 

high  degree  of  interactivity,  coupled  wi  simulation  of  the  fuel  system  prevents  negative 

■•HELP"  system.  The  complete  and  accurate  simulation  of  Jhe 

training,  since  functional  fidelity  is  very  ^  •  allowing  unrestricted  practice  and  then 

also  adds  intaras.  and  ancourages  7"  '^h  o' "  P'Pciont,  since  i,  results  in 
nrr^vidino  corrective  advice.  The  approacn  ib  uui  ^ 

a  high  level  of  student  performance  at  a  very  reasonable  cost. 
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CONCLUSION 


Desert  Storm  proved  that  training  must  be  a  primary  goal  of  military  organizations 
around  the  world.  In  today's  cost  constrained  environment,  however,  the  prudent  tram.ng 
manager  will  look  to  systems  that  are  not  only  cheaper,  but  also  faster,  smaller  and,  .n  many 
instances  better  than  their  predecessors.  These  new  systems  take  advantage  of  technology 
but  also  provide  the  optimal  level  of  fidelity  so  that  students  can  learn  by  doing  at  an 
affordable  cost.  Some  of  their  improvements  come  as  a  result  of  enhanced  capabilities  of 
commercially  available  hardware  -  -  multi-purpose  hardware  that  can  host  a  variety  o 
applications  Others  are  achieved  through  the  application  of  tools  which  yield  development 
eSienc^Ts  The  greatest  cost  and  training  benefits  are  reafeed,  however,  when  these  facto 
are  merged  with  effective  real-time  simulations  of  complex  systems.  Companies  with  the  skiH 
and  technical  expertise  to  do  this  well  can  provide  very  attractive  training  alternatives  to  high 

fidelity  simulators  or  more  traditional  CBT. 
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I .  PROBLEM 


The  process  of  making  good  decisions,  on  any  job,  is  a  complex  mixture  of 
setting  goals,  collecting  information,  defining  alternatives,  weighing  options, 
and  choosing  solutions.  While  rules,  guidelines  and  procedures  can  be  estab¬ 
lished  to  aid  in  this  process  for  a  particular  job,  and  training  for  the  job  can 
explain  and  even  demonstrate  proper  application  of  these,  trainees  seldom  reach 
uniform  proficiency  in  decision-making.  Decision-making  proficiency  is  a  function 
of  knowledge,  skills,  and  rules  acquired  through  varying  operational  experiences. 
In  this  case,  an  attempt  was  made  to  focus  on  the  knowledge  and  decision-making 
acquired  through  "in-vivo"  experimental  training,  (Rasmusson  1986)  as  opposed  to 
rule  or  skill  based  decision  making. 

The  Pacific  Air  Forces  (PACAF)  Crisis  Action  System  (CAS)  is  formed  only  during 
times  of  crisis  or  exercise.  The  job  of  Executive  Officer  for  the  CAS  is  one  of 
assisting  the  CAS  Director  in  managing  the  CAS  and  its  staff  of  up  to  40  or  50 
people.  The  primary  CAS  Exec  duty  is  to  screen  all  incoming  correspondence  (a 
majority  of  which  are  messages)  to  determine  what  action,  if  any,  is  required  of 
the  CAS;  to  assign  that  action  to  one  or  more  staff  members;  and  to  track  the 
progress  of  that  action  to  its  completion,  as  other  related  and  sometimes  overriding 
events  occur . 

The  job  of  PACAF  CAS  Exec  is  filled  by  officers  assigned  to  the  Directorate  of 
Command  and  Control  (DOC)  under  the  Deputy  Chief  of  Staff  for  Operations  (DO) . 
There  is  a  requirement  for  several  people  to  be  available  for  multiple  12 -hour 
CAS  shifts,  which  are  combined  with  other  normal  duty  commitments,  and  are 
subject  to  typical  reassignment  rotations.  This  creates  the  need  for  ongoing 
training  of  new  CAS  Execs.  This,  in  turn,  demands  on-the-job  experience  which 
previously  would  only  be  gained  during  actual  crises  or  large  scale  exercises. 
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II.  APPROACH 


Overview 


The  approach  taken  to  experimentally  address  the  problem  of  providing  CAS  Exec 
trainees  with  realistic  job-like  experience  has  been  to  employ  state-of-the- 
art  off-the-shelf  microcomputer  software  and  hardware  to  develop  individualized 
iob-like  exercises.  Apple  Macintosh  computers  were  chosen  as  the  train^g 
environment  due  to  their  ready  availability  in  the  command  center.  TOe 
exercises  place  CAS  Exec  trainees  in  a  full-color  graphic  environment,  repli¬ 
cating  the  PACAF  CAS.  A  scenario  of  realistic  crisis -related  events  is 
presented.  (See  Figure  1).  The  trainee  is  expected  to  judge  the  significance 
of  each  event,  and  if  CAS  Exec  action  is  appropriate,  to  make  that  decision 
and  take  action  using  on-screen  tools.  These  tools  replicate  real-life  means 
of  decision  execution.  They  include  the  means  for  making  phone  calls,  sending 
messages,  tasking  staff  members,  taking  notes,  logging 

suspenses,  using  references,  and  compiling  situation  reports  (SITREPs) .  ee 
Figure  2  The  trainee  can  also  obtain  advice  and  feedback  on  the  appropriate 
actions  for  each  event.  (See  Figure  3.)  This  is  available  at  varied  expertise 
levels,  depending  on  trainee  experience  and  training  need. 


The  training  system,  called  PACEXTRA  (for  PACAF  CAS  EXec  TOAining)  also 
incorporates  an  interactive  authoring  system  which  allows  Subject  Matter 
Experts  (SMEs)  to  create  and  refine  training  exercises.  This  system  facili¬ 
tates  the  creation  of  individualized  exercises,  composed  of  a  chain  of  custom 
events,  through  a  graphic  interface,  which  requires  no  programming  and  minimum 

training. 


Figure  1  PACEXTRA  puts  CAS  Exec  trainees  in  a  graphic  environment  rep^ 

sentative  of  the  actual  PACAF  Crisis  Action  System.  (Note, 
quality  and  content  of  the  actual  screen  display  was  degraded 
significantly  in  de-colorizing  it  for  this  printing.) 
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Design 


The  design  process  for  PACEXTRA  was  three-phased,  consistent  with  the  three 
major  components  of  the  system.  First,  the  exercise  system  '-.as  designed,  then 
the  authoring  system,  and  finally  the  lessons  All  design  '-.as  constrained  by 
the  chosen  hardware  and  software. 


Design  of  the  Exercise  System 

The  goal  for  the  Exercise  System  was  to  have  it  replicate,  as  realistically  as 
possible,  the  environment  in  which  the  CAS  Exec  performed  his/her  job  in  order 
to  provide  "real"  experience  in  the  events  that  he/she  would  encounter.  The 
desk  and  related  locations  and  items  in  the  CAS  "Ballroom"  were  provided  graphically. 
This  "ballroom"  contains  seven  desk  groupings,  five  of  which  are  hexagonal 
cluster  units.  The  other  two  are  a  set  of  four  administrative  area  desks,  and  a 
modified  "U"  shaped  desk  set  at  which  the  directors  and  executive  officers  sit, 
all  of  which  are  represented  on  the  PACEXTRA  screen. 

It  was  decided  to  use  "Super  3D"  software  to  create  a  three-dimensional  electronic, 
colored  model  of  the  complete  ballroom  complex,  including  furniture  and  pertinent 
equipment,  with  varied  level  of  detail.  Electronic  "snapshots"  of  any  point  in 
the  room  could  then  be  taken  from  any  eye  point  and  at  any  range.  Realistic 
perspective  views  from  the  Exec's  desk  (including  phone,  books,  in-basket, 
terminal,  etc.),  as  well  as  any  location  in  the  room  that  is  pertinent  to  a 
decision  being  made,  were  thus  provided. 

To  present  and  control  these  views,  and  the  variety  of  event- related  items,  it 
was  decided  to  use  the  windows,  fields,  buttons,  and  graphic  objects  inherent  to 
SuperCard.  Thus,  exercise  characters,  their  words  in  cartoon- like  bubbles, 
messages,  phone  conversations,  book  pages,  etc.  would  all  be  developed  as 
overlaying  objects,  with  the  scene  itself  in  the  SuperCard  "background".  This 
approach  would  be  efficient  to  implement,  and  make  it  relatively  easy  to  make 
modifications  as  the  system  matured  and  was  tested.  The  standard  640  X  480 
pixel  Macintosh  screen  was  chosen  so  that  the  exercises  could  be  run  on  any 
Macintosh  II  hardware  with  sufficient  memory  and  speed.  It  was  decided  that  on¬ 
line  aids  would  be  provided  to  introduce  trainees  to  PACEXTRA. 


Design  of  the  Authoring  System 

The  goal  of  the  PACEXTRA  authoring  system  is  to  make  it  possible  for  those 
personnel  supervising  the  training  for  the  CAS  Exec  job  to  create  and  modify 
exercises  with  only  an  hour  or  two  of  training  on  the  use  of  PACEXTRA.  Since 
the  heart  of  an  exercise  is  the  events  and  subsequent  trainee  actions,  it  was 
decided  that  the  authoring  process  would  center  around  a  graphic  "chain  of 
events".  The  chain  is  on  multiple  screen  "pages"  and  is  created  by  linking 
graphic  icons  representing  different  event  types  and  anticipated  actions,  each 
one  labeled  with  its  identifier,  name,  and  source.  (See  Figure  4.)  "Double 
clicking"  individual  icons  on  the  chain  will  then  take  the  author  to  the  same 
screen  that  the  trainee  will  eventually  see  when  that  event  occurred.  The 
author  can  then  add  all  the  details  (in  text  form)  required  to  make  it  a  true 
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Figure  4 .  The  author  works  with  a  graphic  chain  of  events  representing  the 
sequencing  of  the  exercise  and  providing  access  to  the  details  of 
each  event  or  anticipated  action. 

event,  together  with  required  behind-the-scenes  administrative  details.  The 
means  of  making  some  events  dependent  upon  previous  trainee  actions  is 
included  here,  thus  giving  exercises  a  branched  feature.  In  addition,  it  is 
possible  to  define  questions  which  the  trainee  may  ask  of  event  characters  and 
the  possible  responses  the  system  will  give. 

In  order  to  provide  a  realistic  set  of  people  within  the  CAS  with  whom  the 
Exec  interacts,  a  supporting  "cast  of  characters"  authoring  process  was 
created.  The  author  combines  faces  and  bodies  (in  Battle  Dress  Uniforms  (BDU) 
or  flight  suits)  and  assigns  each  to  an  agency  and  a  desk  in  the  ballroom. 
Thus,  a  broad  cross  section  of  rank,  race,  and  sex,  can  be  used  to  staff  the 
CAS  with  a  different  group  of  people  for  each  exercise.  This  approach  permits 
an  author  to  change  or  move  agencies  so  as  to  remain  current  with  the  changing 
real  CAS  makeup,  as  well  as  provide  a  more  interesting  milieu  in  which  the 
student  performs  the  CAS  Exec  duties.  An  on-line  authoring  aid  was  designed 
so  that  exercise  authors  could  learn  about  the  authoring  process  while  on  the 
system.  This,  and  the  guide  for  trainees,  is  available  in  printed  guide 
separate  from  this  report. 
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Design  of  the  first  of  two  exercises  involved  meetings  with  PACAF  DOC  and  DOX 
staff  members.  It  was  decided  that  the  subject  would  be  an  insurrection  in  a 
fictitious  island- country  called  Neptune.  The  events  and  actions  would  involve 
setting  up  a  Joint  Task  Force  (JTF)  and  supporting  the  JTF  with  fighters, 
tankers,  troops,  supplies,  equipment  and  airlift.  The  second  exercise  focuses 
on  orienting  the  new  Exec  to  the  CAS,  through  events  involving  the  staff.  This 
tests  the  application  of  this  system  to  a  slightly  different  form  of  training 
than  the  crisis  action  experience  itself.  (In  practice,  the  trainee  ideally 
goes  through  the  orientation  exercise  first,  followed  by  the  Neptune  exercise.) 

The  exercise  design  process  begins  with  the  development  of  a  "short  story"  on 
paper.  (It  is  important  to  note  here  that  this  first  step  is  ^  a  storyboard 
but  rather  just  a  skeletal  narrative  of  the  events  and  actions  planned.  A 
storyboard  is  inappropriate  because  the  subsequent  authoring  process  will  create 
displays  which  automatically  consist  of  chosen  ballroom  scenes,  people  and  text^ 
To  pre-sketch  them  would  be  wasteful  and  counter  productive.  It  could  be  said 
that  the  on- system  authoring  process  automatically  creates  a  storyboard  as  the 
first  draft  of  the  exercise.)  Once  the  story  is  developed,  the  cast  of  characters 
is  created  on  the  system  and  events  are  sequenced  on  the  on-screen  chain. 
Finally,  the  details  which  reflect  the  "short  story"  are  added.  Thus,  the 
exercise  design  and  development  process  become  one,  once  the  story  is  drafted. 


Development 


The  PACEXTRA  development  process  consisted  of  five  major  steps 

1.  Collection  of  subject  matter  details 

2.  Producing  the  required  graphics 

3.  Developing  the  supporting  software 

4.  Authoring  the  exercises 

5.  Testing,  refining  and  reporting  the  system 
Each  of  these  is  addressed  on  the  following  pages. 


Subject  Matter  Details 

Jn^whTrr  employs,  and  the  environment 

ich  he/she  works  was  obtained  through  visits  to  PACAF/DOC.  Unclassified 
doc^ents  and  forms  were  collected,  measurements  and  photos  were  taken  of  the 
ballroom,  and  discussions  were  conducted  with  staff  members,  particularly  with 

1  qualified  for  CAS  Exec  duty.  As  Che  system  matured,  It  undLwent 
an  informal  formative  evaluation  by  these  people. 


Producing,  the  Required  Graphing 


fwtrnnT.  iT  aT  ballroom  and  its  equipment  were  used  to 

^  principal  CAS  physical  components  to  scale  using  Super 

3D  software.  This  included  the  desks,  chairs,  printers  and  large  screen  displays 
roughout  most  of  the  room,  in  a  moderate  level  of  detail.  At  the  Exec's  desk 

^  refined  level,  including  the  individual  phone  buttons 

and  the  binder  rings  on  clipboards  and  notebooks.  Then  electronic  "snapshots" 

ballroom  screens,  set  up  to  represent  the  Exec's  line  of 

.  ^he  Director,  while 

ned  to  talk  to  visitors,  while  approaching  each  desk  cluster,  and  while 
approaching  an  individual  cluster  desk  position.  Capture  (TM)  software  was  used 

to  take  the  electronic  snapshots  which  were  in  turn  imported  into  the  SuperCard 
environment  as  card  backgrounds.  peruara 

Super  3D  was  also  used  to  create  enlarged  views  of  the  Exec's  tools,  such  as  the 

notebooks,  message  clipboards,  computer 
minal,  ^c.  These  were  captured  and  imported  into  SuperCard  as  graphic 
°  u  u  ^  graphic  functions  built  into  SuperCard  were  used  to  modify  and 

embellish  these  items  and  to  create  other  items  such  as  the  cartoon- like  bubbles 
used  to  enclose  the  words  spoken  by  scenario  characters. 

To  support  the  process  of  creating  a  cast  of  characters,  a  graphic  artist  drew 
monochrome  faces  and  bodies  using  Super  Paint.  These  were  imported  to  SuperCard 
where  they  were  sized,  colored,  and  refined  so  as  to  create  a  library  of  some  24 
faces  and  12  bodies.  These  include  male,  female,  black,  white,  officer  (Major) 
enlisted  (MSgt.),  BDU  and  flight  suit.  The  author  has  access  to  these  on  pallets 
while  using  the  "Casting  of  Characters”  feature. 
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The  drawing  and  painting  feature  of  SuperCard  (in  some  cases  aided  by  Super  3D) 
were  used  to  create  the  remaining  graphics.  These  include  the  icons  and  background 
for  the  event  chain,  large  "Help",  "Stop"  and  "Next/Past  Event"  buttons  for  the 
trainee,  and  other  items  on  behind-the-scenes  authoring  screens.  SuperCard's 
buttons  and  fields  were  used  extensively  in  this  area. 


Developing  the  Supporting  Software 

As  explained  previously,  a  great  deal  of  the  development  of  PACEXTRA  involved 
using  the  non- programming  features  of  Supercard  to  create  windows  and  cards,  to 
draw  and  paint  graphics,  and  to  define  and  insert  buttons  and  fields.  However, 
these  hypermedia  functions  alone  are  only  a  portion  of  PACEXTRA.  To  make  the 
system  perform  as  it  does,  a  significant  amount  of  software  code  (Supercard 
script)  was  written.  Approximately  1,000  hours  of  programmer's  time  was  required 
for  this . 

Due  to  the  object-oriented  nature  of  SuperCard,  the  unique  PACEXTRA  code  (script) 
is  contained  in  the  various  system  objects  including  projects,  windows,  backgrounds, 
cards,  fields,  buttons,  and  graphic  objects.  PACEXTRA  Software  Document  (separate 
from  this  report)  describes  the  software  scheme  in  some  detail.  The  script 
itself  is  accessible  within  PACEXTRA  through  SuperCard  and  its  subsystem,  SuperEdit. 


Authoring  the  Exercises 

The  two  exercises  were  developed  by  first  outlining  a  series  of  events  on  paper 
(in  much  the  way  a  short  story  might  be  outlined).  A  cast  of  characters  was 
then  developed  with  which  to  staff  the  CAS.  A  chain  of  events  and  anticipated 
actions  (responses  by  trainees)  was  developed  on  the  system,  and  details  were 
added  to  each  event  and  action.  These  details  included  the  text  of  messages, 
phone  calls ,  and  direct  contact  with  CAS  characters .  They  also  included  the 
definition  of  any  conditions  (trainee  actions)  which  are  to  influence  the  occurrence 
of  specific  events.  The  result  of  this  process,  and  the  reviews  and  refinements 
which  followed,  are  the  two  exercises  delivered  with  PACEXTRA. 

The  scope  of  this  project  did  not  allow  for  formal  testing  of  PACEXTRA.  Testing 
has  been  limited  to  that  done  by  Logicon  during  development,  and  as  a  result  of 
using  the  authoring  system  to  develop  the  two  delivered  exercises.  Informal 
formative  evaluations  were  conducted  with  the  PACAF/DOC  staff. 
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Ill .  RESULTS 


This  project  resulted  in  the  production  of  an  experimental  prototype  of  a 
microcomputer-based  training  system  called  PACEXTRA.  It  runs  on  a  Macintosh  II 
computer  and  provides  individualized  exercise -based  training  to  those  preparing 
to  perform  duty  as  the  Executive  Officer  in  the  PACAF  Crisis  Action  System.  The 
system  demonstrates  the  concept  of  using  low  cost  hypermedia  software  (S  ioerCard) 
to  create  a  user-friendly  exercise  authoring  tool,  the  implementation  of  which 
results  in  realistic  event-driven  exercises.  These  exercises  put  the  trainee 
into  a  highly  graphic  on-screen  environment,  which  is  expected  to  provide  high 
transfer  of  training  to  the  real  job  environment.  Exercises  challenge  the 
trainee  with  events  about  which  he  must  make  decisions  regarding  action  to  be 
taken.  The  system  provides  four  levels  of  advice  and  the  means  of  taking  action 
in  realistic  job-like  ways  such  as  sending  messages  or  contacting  others,  either 
directly  or  by  phone.  Feedback  on  what  was  expected  and  whether  it  was  accomplished 
are  provided  for  each  event. 

This  project  focused  only  on  the  design  and  development  of  the  training  system 
and  included  no  evaluation  or  research  on  its  effectiveness.  Any  such  future 
research  by  AFHRL  or  others  will  be  reported  separately.  . 


Advantages  and  Disadvantages  of  Approach 


The  advantages  of  employing  a  low  cost,  off-the-shelf,  object-oriented,  hyper¬ 
media-based  software  package  like  SuperCard  is  that  it  provides  a  great  many  of 
the  features  needed  for  interactive  training  systems,  yet  leaves  the  application 
of  these  features  up  to  the  imagination  and  ingenuity  of  the  training  system 
designer/developer .  The  ability  to  create,  position,  color,  re -size,  and  add 
functionality  to  windows,  cards,  buttons,  fields  and  objects  without  program¬ 
ming,  and  to  then  further  enhance  their  functions  with  scripts,  gives  the  designer/ 
developer  an  ideal  tool.  However,  the  scripting  language,  due  to  its  interpretive 
nature,  runs  very  slowly,  particularly  on  less  than  the  top-of - the  - 1 ine'  Macintosh 
hardware  (fx  model). 

The  true  advantages  and  disadvantages  of  PACEXTRA  as  a  training  system  will  not 
be  known  until  it  is  tested  and  employed  in  the  field,  which  is  beyond  the  scope 
of  this  project.  However,  experience  in  applying  the  PACEXTRA  exercise  authoring 
system  has  shown  it  to  be  very  easy  and  efficient  to  use. 
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Potential  Application 


PACEXTRA  itself,  because  it  was  designed  specifically  to  train  only  the  CAS  Exec 
at  PACAF,  has  no  other  potential  direct  application.  However,  the  approach  to 
event-driven  exercise-based  training  which  it  exemplifies  has  application  to 
nearly  any  event- related  job.  For  example,  it  could  be  used  to  train  store 
clerks  on  how  to  handle  various  customer  situations.  On  the  other  end  of  the 
spectrum,  it  could  be  used  to  give  space  crews  pseudo  experience  with  mission 
events  which  necessitate  their  making  decisions  and  taking  actions.  To  convert 
PACpCTRA  to  train  another  job  would  primarily  involve  creating  the  proper  graphic 
environment  and  the  on-the-job  tools.  The  underlying  PACEXTRA  system  has  generic 
application  potential. 

The  real  potential  of  the  type  of  training  system  which  PACEXTRA  represents  is 
to  teach  both  procedural  or  rule-based  decision  making  and  knowledge -based 
decision  making,  particularly  because  it  provides  so  much  of  the  complex  context 
of  a  decision  environment.  This  is  in  contrast  to  procedural  trainers  which 
simply  list  "rules"  or  procedures  with  little  sense  of  the  "knowledge"  that  is 
gained  by  having  experience  in  the  decision  context.  This  is  the  force  behind 
expensive  "virtual  world"  devices.  The  advantage  of  the  PACEXTRA  approach  is 
low  cost,  availability,  the  focus  on  coordination  of  activities  (i.e.,  teamwork), 
and  speed  of  exercise  authoring.  There  is  little  evidence  to  suggest  the  true 
need  for  the  glamor'  of  high  tech  reality.  What  is  needed  is  the  work- inspiring 
nature  of  an  interactive  environment.  PACEXTRA  appears  to  provide  this,  at  low 
cost  and  with  minimal  effort  on  the  part  of  scenario  authors.  Its  potential 
seems  to  be  significant. 
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Recommended  Follow-on 


As  described  elsewhere  in  the  report,  the  event-driven  exercise  format  of 
PACEXTRA  has  innumerable  other  potential  applications,  and  the  authoring  system 
which  it  incorporates  makes  it  relatively  easy  for  trainers  to  create  and  modify 
training  exercises.  A  natural  follow-on,  therefore,  would  be  to  modify  the 
system  (primarily  the  authoring  system)  so  it  focused  upon  another  job  and  its 
environment,  so  that  exercises  could  then  be  created  to  train  that  other  job. 


However,  an  even  more  useful  follow-on  would  be  to  experiment  with  developing  a 
higher  level  "authoring"  process  which  would  permit  a  training  system  developer 
to  create  their  own  exercise  authoring  system  for  their  unique  job.  This  higher 
level  authoring  process  could  then  permit  the  exercise-based  training  exemplified 
in  PACEXTRA  to  be  applied  to  training  any  other  job,  without  going  through  the 
time  and  expense  of  building  the  supporting  tailored  authoring  system  through 
programming.  Rather,  the  trainers  could  develop  their  tailored  authoring  system 
themselves,  through  an  interactive  process  similar  to  that  of  other  interactive 
Macintosh  applications.  The  result  could  add  another  dimension  to  the  media  and 
methods  available  in  technical  and  operational  training  throughout  the  Air 
Force,  and  elsewhere.  (Note:  With  the  advent  of  Windows  3.0,  and  object  oriented 


hypermedia  applications  such  as  Toolbook,  this  approach  now  appears  practical  on 
the  IBM  PC  and  compatible  family  of  computers  as  well.)^ 


'■Rasmussen,  J.  Information  Processins  and  Human-Machine  Interaction. 


Vol.  12,  North-Holland,  NY,  1986. 
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afidad  tx3  tl^  Standard  hardware  was  i 
available  for  aJrait  $400 « 


tter  was  considenKi  a  promising  rqplaoertent 
am.  UhlUce  the  HP  9836,  this  cott^ter  had 
^  the  Navy,  was  alHost  always  equipped  with 
$1,100”$!, 500,  vicB  $14,000  for  the  9836. 

5,  run  at  12MHz  clock  speiKi,  and  support  EGA 
th^e  ocaiputers,  all  tiiat  would  have  to  be 
a  mth  ca^arooessor  and  a  mouse,  both 


Tte  ssotmd  decisicm  was  to  cteJ^a  a  prograimnir^^lai^uage.  The  ”C’' 
language  was  chosen  li^^iiuse  it  was  ii^ll  siit^  to  simulation  applications, 
su^iorted  the  modularity  neiKisS,  ar^  pos^d  no  partioilar  problem  in  blerKiir^ 
text  arKi  gr^pHir-  pr^entation.  ?fe^t  inportantly,  translation  frcm  B^IC  to  C 
was  p^sible.  (The  original  P”3C  HEIA  software  routines  had  b^n  written  in 
MSIC  and  w^re  adsmtad  to  HP  MSIC  for  hosting  on  the  9836.) 


practicality,  nany  of  the  suggestics^  received  frcan  the  users.  A  c^ta  base  o 
these  sugg^tiCTis  h^d  i^en  maintained,  and  would  eventually  prove  invaluable 
toward  increasing  the  training  value  of  the  fields  program.  Bsoause^the  P~3 
WING  training  persOTnel  frequently  rotate  and  transition  to  C3th^  assigrm^ts 
the  develcpxent  gdso  agreed  bo  another  rourai  of  cSesign  revi®^  it^tu^. 

I  '  ' 

F 


^  the  "DIC"  series,  is  d^ignatei  for  tac^tical  usage,  such  as  tactical 
d^ision  aids,  large  data  base  intensive  applicaticrss,  and  warfare  analysis 
ard  er^gement  iKdelir^.  The  other  standard  desktop  compu^  is  knwon  as  the 
“tesJctro”  seri^,  by  the  military  for  routine  procass3J^  aj^lications, 

and  is  v^y  similar  cr  icSentical  to  cxnsiom  PCs  us«i  in  offices  ard  hemes.  At 
the  of  this  writirg,  the  current  standards  are  the  Sun  Microsystens  DTC 
II,  and  the  UNISYS  386  C^ktep  III. 
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icml  DescriptiOTi 


-  in  les.  ona  year, 

Structured  as  shewn  in  Figure  1. 

^  desioned  to  furnish  botn 

-Hie  TOTORIAL  module  ^^Minewhat  more  detailed  knowledge 

basic  badcground  for  the  its  host  platform, 

of  weapon  available  which  provide  dynamic 

^  eraa^le  o*  tne  leamin,  ^ 

bv  describir^  the  operation  of  ^=5!!L?Ls  throucii  a  mouse  click  or  depression 
ibe  student  has  access  to  these  funct  learned  in  ascending 

^a^SSe  softkey.  Ibe  lessons  ^  SWEEP,  a 

Irter  S^the  TOBMM.  Main  Menu  drawn  for  the  sbxJent 

finS-e  cycle  of  the  Harpoon  ra^  lesson  ^student  learns  that  if  ^ 
in  vivid  color  aoquisition  of  the  target  ^ 

searcii  begins  close  to  two  t^g^  infSmtion  on  the 

the  right  is  more  likely.  Detailea  rames  swe^  angles,  and  how  range 

such  as  miniirm  -n^e  PATIFPN 

apertures  are  seartoed  witl^  ,LSSs  tS^veloproent  of  a  Harpoon  searto 
development  lesson,  [F3],  ^  use  of  different  varianto  of 

pattern,  and  the  variations  target.  Many  combinati^  of 

S^ile,  searto  P^«^,^i;jtr^^4^1ierent  pattern  se^ions, 
these  parameters  are  possible,  s^  ^  patterns.  The 

3  missile  variants,  iJTthe  tactical  use  of  the  weapon.  _niis 

search  options  have  nuch  sigr^f  as  a  long  range  air  ^ 

platform,  is  a  rather  slow,  ranges  and  minimum  detectabili^  o 

student  will  be  prcnpted  ^  .  d^h  the  searto  mecharasm  pictorially 

vAien  he  witnesses  the  xoissile  -  particular  target,  or  land  mass, 

flayed,  the  tea^  ^  DJVEWrMEHT  =hd  T^ET 

will  beoate  eiCTption^ly  =1^;  £U^;  the  fomer  to  pattern 

(or  la^I,  intact. 

T^,  the  student 

at  tJr^lhnlng  interest^ 

n?u^jS°Si?tS  ^  i-ediately  proceed 

ENGAGTMENT  U^AINING  module. 
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^  an  TVFPfCISE  GENERATOR,  an  EXERCISE 

A  me  Hm  program 

^KMUIAK® 


_  me  r^aCEHEOT  TBMKQG  <==^^ 

a  rassnz  SIMUUaOR.  scenarios.  Tme 

^S'sSfication,  <=an  ra  sim.^  on 

<jepth  of  ^Selling  and  nunte^  deg^es^  ^  program. 

S-^TcaSouSS^  *  is  sn^n  ra  3. 

po.ry^«>  Qf  Values  Default 


—  Ek_—  CONFIDENnAL  OONFIDENnAL 

Classifi-  Places  ^  SECRET 

cation  the  tactical  display  area 


Land 


Allows  selection  of  preprogranned 

SSLses, 

rent  or  stored  landmasses, 
Station  of  coastline  on  TAOCO  s 

di^lay- 


No  Land 
land 


No  Land 


Contacts 


lude:  selection,  raioval  Launch  mode) 

tact  ®2neuvers,  contoct  contacts 

ESM  emission  sciiedule,  oonta 
library  manager  (database) 


No 

Contacts 


_  a'Hi—  Launch  Platform  Launch 

Platforms  PI*®  mar  only- 

ributes  {radar,  Platform 

altitude,  turn  rate,  accel/oe^ 

rates, 

participatiig  unit  {list  of  20), 

ssriitS^^r<tsSm 

or  TBDdify  attriJwtes  of  platform 
and  sensors  in  library} 


OttUmy’' 

Launch  Platform 
and 

Participating 
Unit 


Scenario 

Pages 


parameters  {surface  wiro 
Sft  wind  spd/<^,  ambi^  ^ 
tenperature,  ram 

SimLation  control:  {start  , 

^  of  visibility,  grid  1^  rrg/ 

gS^rrors,  aircraft  «3Ui^ 
2^'gsts  radar  /®' 

l^Le  capabiliW- 

out  {allows  loadout  of 
missiles  on  wing  weapon  stations} 


Weather: 

Wind  O—lOO  )cts 
Tenp  -16"120®F 
Sea  State:  0- 
15  Douglas 

0-  iran/hr  No  Ram 


0  kts 

59  op 

2  Douglas 


Rain: 


Neapcn  loadout: 
0-6  missiles 
Al3<-84A,C,or  D 


6  AC31-84D 
missiles 


Brief 


_  .  v<»-ioe*  a  textual  desc”  0”13 

ths  mission  ^^ves 

BlanKSURPIC- 


lines  text  No  text 


Blank 


^jsed  b/  P-3  cotnunxty 


DXCUlTw 

Conplete  SURPIC  SURPIC 
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<  ^  t-im  •the  scenario  created  or  recalled 

rt^  EXEBaSE  ^le  plac^  the 

fron  menory  in  the  ^000  ^  he  assumes  responsibility 

student  in  the  role  a  ^  to  develc^ 

oontrollir^  the  launch  platfc^»  qp^  ^  participating  unit) , 

targeting  data  (and/or  using  the  engagement  by  launching  one 

Harpoon  engagement,  and  exeaiting 

or  more  missiles. 

Mere 

^■iac  htadraroe.to  as  in  the  aircraft,  is 

Of  flyim  the  aircraft.  Route  oon^i,  /steering  ocnputations  are  performed 
throi^  the  insertion  of  steering  comma^ 

ky  the  aircraft's  tactical  cajpu^to^ro^  actually  two  dimensioral 

^  SrSlSSS;  JSisver.  »st  oanuauy  entared 

of  a  PC  softkey  function. 

rhB  EXERCISE  SIMUIATCfR  .  Jf ^SSt to  main 

of  the  switch  legends  m  the  ^  labeled  TRACK  at  the  mam  menu 

mem  subgnxping.  ^ive  switch  legends  change  as  follows: 

level.  If  [FI]  IS  selecte^^l  of  rMl=M3VING  CIRCLE, 

[^J^GENERATE  IRACK,  [F31==^’?^^L5^^0n7  [E7]*FIX  DESIGN^, 
CF5]=OQMPUrE  mrERSBCnON,  [F6]  pry  rF10]=^totum  (capitalized 

S  S^ility  of  i'l^^S'iStically  to  TACOO  fusions 

^^ons  are  in  the  same  ^^tim  ^  S^sSSSed  1^30  TACOO  functions 
aircraft) .  SStSld,  typical  task  flew  will 
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r^ity- 
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<■>  o  □  □ 


f:irVBO]lM£NT] [fire  CONTROL] |CONTAI 


LEGO® 

4-  -  Hostile 
®  ~  Friendly 
1  “  Heutrsl 
^  -  Unknown 


I  • -  Searoh 


SSt^unrS^riii^ISleSc^^ile  have  failed  to  itpa^ir 
great  vy.ue  in  oetiermiju^  ^  valuable  functions  in  HEIA. 

ffS^^Sn^^S^are  ihcluaed  which  allM  tte  stud^  to  tolt  and 

Mttoletow  at  any  time.  Oie  Paoq  functicn  gives  a  ^le  ^ 

tar  all  of  the  missiles  in  the  exercise,  and 
for  Uunch,  seeker  turn  Ch, 

outcome  for  all  missiles.  It  also  gives  ^  fairly  c^prehers^^sgay^^ 
printout  of  all  of  the  pertinent  engaganent  variables,  and  revea 
identity  of  all  of  the  contacts  in  the  exercise. 

Cost  Effectiveness 

EPr  a  program  capable  of  such  detailed  engaganent 

L?^^r4fafc^c*eaD  to  produce.  Software  development  costs  ran  about 

$2^  000  management  and  travel  about  $100,000,  and  life 

^  to^Sn  the  prtjgram  for  4  additional  years  about 

SStog^^-I^Cansys  386)  am  1*^  fislded  to 

^  these 

since  Hm  will  ^ 

^nr  Fleet  use  can  be  distributed  widely  for  training  at  excraneiy 
Ihe  SSoanarit  team  usually  conducts  turnover  training  “ 
SgtgSTWam's  capabilities,  particularly  these  found  in 

the  EXERCISE  GESffiRATOR. 

Other  Bigagement  Training  AixJs 

Asicie  from  the  P-3C  HEEA,  a  number  of  other  engagemert 
been^^SSloped  andl  several  others  are  under  current 
Engagement^aining,  variants  of  the  Hm 

SpT^virw  aircr^St  and  the  A-6E  Intruder  aircraft.  Over  40  Hm  training 

systems  are  currently  in  use  Jy  tSiSbs^  has 

JLr-.’iar-c  AfUitionallv.  a  program  cadled  M  on  N  (W  snoocers  on  « 

been  developed  to  handle  the  pre-laundi  analysis  and  po^launch 
SgnstructiS  Of  a  a^  ot 

SeSl^  S^cSigliTSg-sSff^ve. 

T«  *vji<-ion  to  these  variants  of  HEIA,  future  versions  of  the  program  will 
I.  dSL^gS^Srgi^fS  S^TiSnehsd  Harpom,  sulmmrina  laun^  Harp^, 

SSS;?  a  gre^  based 

HEIA  system. 
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Cmpjter  i^ssd  engagotent  training  prograns  have  also  teen  develcpal  for 
the  laser  guidei  versim  of  tiie  Jfeveric^  missile  (A£3i~65E) ,  arrf  Bigeye 
Qiemiml  (BI1J-80/B) .  Hie  latter  weapon  prcgram  was  later  cancelled  in 


infrareJ  versiOT  of  Mavi^ick  {MM-65F) ,  aoid  'tiie  Mid-Range  and  Bferitine 
IMnarr^  Air  Vi^iicli^o 


SunnBry 


Ocapiter  bas^  s-gagocasTit  trainers  such  as  the  P-3C  HEEA  fill  an  irrportant 


stiiderst  with  tte  pc^  lauiKh  of  the  d^isions  made  in  the 
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SLroE  1 


I.  Introductioo 

one  often  hears  ft.,  models  wiU  he  """  °Sde“ 
tested  and  validaffid,  ftie  “  "“tS’te  designer  actually  means  is  that  mote  data  is 

rerr  srr  ^^rr^hse^^ 

in  1986,  KETRON  was  conftac^  “  f we« 

infrared  variants  of  the  Mavenck  jcguisiuon  within  the  training  community.  To 

not  available  to  simulate  electro-optic  energy  ^  "Bootstrapping"  applied  to 

^o^«“:^rstrre::;:i^ 

economy  of  effort  ft)  keep  the  modelling  cost  down. 

SLIDE  2 

The  purpose  of  ftus  paper  “ “”3“Xng‘^d“Tme 
modeUing  used  in  the  development  °f  complexity  in  simulating  EO 

Maverick  missiles.  This  paper  ts  ■"““va^  *"  - '„eLt  to  underscore  the 

engagements  on  a  microcomputer.  nnnrt  of  PC-based  training  devices.  In  this  regard 

importance  of  validation  in  the  h  ecyc  ®  .  jj  established  practices  of  statistics  and 
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III.  Scope 


Before  discussing  our  approach,  a  brief  word  on  the  scope  and  importance  of  EO 
modeling  techniques  to  the  DOD  is  an  appropnate  prologue. 


As  shown  on  the  slide  a  considerable  number  of  NAVY  weapons  depend  on  EO 
sensors  for  detection,  acquisition  and  tracking. 

(Slide  4)  Laser  Designators  and  Laser-guided  Weapons 

-MAVERICK  Missile  (TV\Infrared\Laser) 

-TOW 

-WALLEYE 

-HELLFIRE 

-SLAM 

-Shipboard  Infrared  Search  and  Track  (IRST)  system 
-Laser  Designators  (MULE\LDT) 

-SIDEARM 

In  view  of  the  DOD  investment  in  EO  weapons  by  the  DOD,  we  ^su^  at  start 
of  the  effort  that  we  could  find  an  EO  model  that  had  been  validat^  by  the  DOD  Md  adapt 
i,  to  Mavl^idc  At  a  minimum,  we  needed  access  to  a  Maverick  database  to  develop 
empirical  models  for  predicting  performance. 

We  found  out  that  in  the  late  1970’s,  US  DOD  directed  the  joint  «rvices  to  provide  a 
capabiUty  by  1984  to  measure,  model  and  predict  accurately  the  atmosphenc  transmission 
e^ts  o^DOD  sensors  and  communications  system.  The  roles  were  assigned  as  follows. 


SLffiE  5 
a.  Army; 

Measure  and  model  atmospheric  propagation  conditions  to  battlefield  conditions.  Develop 
models  describing  natural  and  artificial  dust,  smoke  and  chemicals  and  their  propaganon 
chmcteristics.  ^elop  batdefield  diffusion  models  to  relate  propagation  conditions  to 

meteorology. 

SLIDE  6 
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b.  Air  Force: 


Maintain  approved  DOD  standard  atmospheric  optical  models.  Publish  and  brief  model 
updates  annually.  Conduct  a  tri-service  annual  review  conference  to  provide  for  tri-service 
discussion  of  model  deficiencies  and  model  propagation  effects  of  the  free  atmosphere, 
including  the  ionized  atmosphere  above  50  km. 


c.  Navy: 

Measure  and  model  atmospheric  propagation  conditions  as  affected  by  the  marine 
environment. 


d.  All  Services: 

®  Develop  relationships  between  meteorological  measurements  and  aerosol 
models. 

®  Conduct  adequate  meteorological  and  propagation  measurements  in  conjunction 
with  systems  tests. 


®  Develop  and  validate  systems  performance  models  based  on  weather 
climatologies. 


The  most  widely  acclaimed  EO  model,  LOWTRAN,  is  owned  by  the  Air  Force  and 
runs  on  a  main  frame  computer  located  at  Massachusetts  site  of  the  Air  Force  Geophysics 
Laboratory,  now  known  as  Phillips  Laboratory/Geophysics  Directorate  (PL/GP).  Accessing 
LOWTRAN  from  a  mam  frame  computer  was  impractical  for  META  considering  the  large 
amount  of  access  required.  Moreover,  documentation  on  the  model  inputs  suggested  that 
attempting  to  write  code  for  the  complex  models  used  in  LOWTRAN  was  beyond  the  scope 
of  what  w^  required  for  a  training  device,  especially  in  the  prototyping  stage  of 


SLIDE  9 


We  discovered  however  that  a  considerable  effort  was  underway  to  convert 
LOWTRAN  into  a  Zenith  248  format  for  the  purpose  of  supporting  PC-based  EO  Tactical 
Decision  Aids. 
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Ahown  in  me  list  o^EO  ^  totn",  weT^  »'S!fn'he"riluc 

’•boouSpping^pVoa"^'  developing  EO  acquisition  models  for  ME 


c;^t>4MARY  of  EO  MQDELS, 

,.nn  pn  Models  HQSIXQMEIIIEE  AfiElJI 


•lowtran 

•MODTRAN 
•SENTRAN 
•PCTRAN 
•  MKni  EOTDA 


Main  Frame 

It  »  '* 

It  ti  « 

Microcomputer 

Microcomputer 


Air  Force 
Air  Force 
Germany 
Air  Force 
Air  Force 


IQ£ 

Early  1980’ s 
Early  1980’ s 
Early  1980’ s 
Late  1980’ s 
Early  1990’ s 
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IV.  Dlscissslon 

We  will  now  summarize  our  "bootstrapping"  process  across  four  development  penods 
or  phases: 


SLIDE  1© 


Phase  I  (1987-1989)  Empirical  Methods  based  on  Laser  Maverick  OT  Data 
Phase  11(1989-1990)  Theoretical  Methods  based  on  Hughes  Model 
Phase  111(1990-1991)  PC-based  Software  Investigation 
Phase  IV  (1991-1992)  MK  ffl  EOTDA  and  Real  World  Data 


A.  t  niW7-l989>:  Prolaigy  AV-SB  Um  MECA 


Collecting  operational  data  is  the  essential  first  step  in  the  "Bootstrapping"  procedure 
in  that  the  data,  however  small,  inherently  will  account  for 

uncertainties  of  human  and  mechanical  factors.  A  summary  of  the  data  made  available  to  us 

at  the  start  of  the  effort  is  shown  on  the  next  slides. 


Data  on  EO  designation  range  versus  operator  pointing  error  and  dedgnator  to  target 
were  obtained  through  the  NAVY  Maverick  Program  Office  (PMA  242). 


These  data  points  were  supplemented  by  Commander,  Operational  Testing  and 
Evaluation  Forces  (OPTEVFOR)  data  on  Laser  Maverick  launches  for  the  A-4E  and  A-6E 
du^^Mti Wtion  test  and  evaluation  (lOT&E).  A  total  of  51  l^r  Mavenck  launches 
from  A-4M  A-6E,  and  AV-8B  aircraft  were  used  to  correlate  ground  designaUon  and 
Maverick  siker  acquisition  range  with  visibility,  target  type,  aircraft  altitude,  and  target 

range. 


SLIDE  14 

The  second  step  of  the  "bootstrapping"  approach  is  the  development  of  heunstic 
algorithms  to  be  used  in  computer  simulation.  In  this  regard,  regression  eq^tions  were 
derived  for  each  set  of  parameters  identified  in  the  operational  data  to  test  the  dependencies 

^“rohitonSips.  R-sqLcd  estimamrs,  .-value.,  aud  f 

to  verify  that  the  equations  and  their  variables  were  staUsUcaUy  significant.  The  residuals  of 
the  regressions  were  examined  for  patterns  which  could  possibly  improve  the  equauons.  The 
Durbin-Watson  Test  for  each  regression  was  performed  and 

autocorrelation.  Finally,  a  randomized  block  analysis  of  v^ance  (;^NOVA)  w^  ^rformed 
to  investigate  the  variability  of  each  treatment’s  mean  and  the  vanability  of  each  block. 
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Using  these  techniques,  a  simple  relationship  was  developed  to  associate  Maverick 
seeker  acquisition  range  with  designation  range. 

(SLIDE  15,  16)  Show  plots  of  the  data  and  the  predictive  equation 
Lessons  Learned  in  Phase  ! 

The  results  of  Phase  I  modelling  lead  to  the  following  conclusions; 

(SLIDE  17) 

•  "Bootstrapping"  methods  applied  to  operational  data  are  very  cost  effective  in 
training  system  simulation 

•  "Bootstrapping"  results  are  sufficient  estimates  of  upper  and  lower  bounds  on 
EO  sensor  performance 

•  "Bootstrapping"  algorithms  are  not  suited  for  EO  training  courseware  i.e. 
derived  equations  represent  the  data  only,  not  the  physical  phenomena 
involved. 
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Introduction 


Table  1.  Variations  in  Meteorological  Range 
(Bad  Weather) 


Weather 

Mf^tmroloffical  Range  (mefgrsl 

Dense  fog 

<50 

Thick  fog 

50  -  200 

Moderate  fog 

200  -  500 

Light  fog 

500  -  1000 

Thin  fog 

1  -  2  km 

Table  2.  Variations  in  Meteorological  Range 
(Good  Weather) 

Weather 

Mptftorolopical  Range  ddlo-melersl 

Metric 

Clear 

10-20 

Very  clear 

20-50 

Exceptionally  Clear 

>50 

C  Phase  in  f1990-l991V-  Infrared  Modeling  fpr  F/A-,18  META  AppligahPilS 

In  early  1989  as  we  began  to  transition  from  laser  to  infrared  modeling,  the  Air  Fo^ 
beean  to  make  LOWTRAN  output  available  to  the  DOD  community  in  the  form  of  - 

called  the  PCTRAN  series.  With  the  aid  of  on-line  LOWTR^  output,  we  would 
be  ile  to  expand  our  modelling  parameters  and  extend  META  potential  m  the  area  of 
mission  planning  and  rehearsal. 
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Algonthm  1.0 

ES  =  DP  *  PL 


TRANSMISSION  TO  TARGET 
SLffiE  39 
Inputs 

SL  =  Slant  Range  (feet) 

VI  =  Visibility  (or  weather  condition;  in  this  case,  it  (feet)  would  be  converted  to  a 
visibility  number) 

AL  =  Altitude  of  Airplane  (feet) 

Output 

TR  =  Transmittance  Distance  (feet) 
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Algorithm  2.0 

TRANSMITTANCE  EQUATIONS 


VisibiltoJkm)  Equation 

+  0.90 
+  0.81 
+  0.44- 
+  0.115 


ENERGY  REFLECTED  BACK  TO  DESIGNATOR 


23 

15 


TR  = 
TR  = 
TR= 
TR  = 


-0.042SL 
-0.048SL 
-0.038SL 
-0.01  ISL 


lasMis 

ET  =  Energy  Transmitted  to  Target 
ES  =  Designation  Energy  at  Start 
TR  =  Transmittance 
SO  =  Energy  Spillover 
EA  =  Energy  Absorbed 

Based  on  the  above  inputs,  we  derive  the  following  EO  model  for  Phase  III: 
SLIDE  42 


Algorithm  3.0 

ER  =  ESTR  -  (SO=*-ET)  -  (EA-^(ET-(SO^ET))) 

ER  is  the  energy  in  millijoules  which  is  being  reflected  back  towards  the  laser 
designator. 

AREA  OF  ENERGY  CONCENTRATION  AT  SEEKER 

The  energy  concentration  at  the  seeker  is  modeled  as  the  energy  per  square  meter  of 
area  over  which  the  laser  beam  is  spread.  During  designation,  the  beam  is  coherent  and  thus 
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the  energy  is  very  concentrated  upon  its  arrival  at  the  target.  But  when  the  laser  makes 
contact  with  the  target,  it  is  reflected  back  in  many  directions,  spreading  the  energy  out. 

(SLIDE  43):  Discuss  the  Energy  Concentration  Model: 

To  calculate  the  energy  concentration  at  the  seeker,  we  need  to  model  the  area  which 
the  energy  is  spread  over. 

AREA  OF  ENERGY  CONCENTRATION  AT  SEEKER 
Inputs 

SS  =  Spot  Size  (inches) 

SL  =  Slant  Range  (km) 

Output 

AC  =  Area  of  Concentration 
at  Seeker 

SLIDE  44  (Discuss  Picture) 

where  AC  is  calculated  as  follows: -  - 

Algorithm  4.0 


_  55*2.54. 

100(2)  ^ 

R=(SL*lOOO)+r 

AC=itR^  [Square  meters] 


energy  concentration  at  seeker 


SLIDE  45 


Inputs 

ER  =  Energy  Reflected  Back  (millijoules) 
AC  =  Area  of  Concentration  (square  meters) 
TR  =  Transmittance  Calculation 
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DETERMINE  IF  ACQUISITION  OCCURS  AS  PREDICTED  IN  PHASE  I 
SLIDE  47 

Iiieum 

SE  =  Seeker  Sensitivity  (millijoules  per  square  meter) 

EC  =  Energy  Concentration  at  Seeker 

Note:  For  Laser  Maverick,  assume  a  sensitivity  betv(/een  2. IX  and 

10.2X 

Qu®ut 

EC  ><  SE 

Algorithm  6.0 

If  EC  >  SE  then  acquisition  is  jK)ssible.  Otherwise,  declare  acquisition  is  not 
possible. 


STATISTICAL  COMPARISON  OF  LOWTRAN  WITH  PHASE  DATA 

Phase  I  operational  data  showed  a  somewhat  surprising  closeness  with 
LOWTRAN  data  output  as  shown  by  the  following  regression  equation: 
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SLIDE  48 


LOWTRAN  Acquisidou  Range  (kft)  =  1.1811  +  (.8369)  X  Phase  I  Range  (kfl) 

qtadsdcal  analysis  of  the  tiata  determined  an  R-squared  value  of  0.8763  and  a 
standard  deviation  of  8.4188  (kft).  In  statistical  parlance,  we  <^nclude  on  me  ton,  of 
these  low  values  that  LOWTRAN  and  Phase  I  operanonal  rraults  are  not  signifi^Uy 
tot.  Furthermore,  these  tesults  imply  that  LOWTRAN  results  represent  upper 

bounds  on  EO  performance. 

Summary  of  Phase  III  T  ^SSQHS  Lgani^l 


SLIDE  49 

•  Greater  confidence  was  established  in  META  results  obtained  in  Phase  I 

•  Accuracy  of  Phase  II  "Bootstrapping"  approach  depends  on  the  number  of 
equations  used  to  predict  transmittance 

•  LOWTRAN  parameters  would  be  preferred  for  Mission  Planning  and  Tactical 
Decision-making 
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D.  PHASE  IV  (1991 -1997V  ('Development  of  A-6E  Laser/ Infrared  META) 

Currently,  we  are  in  Phase  IV  of  development  and  are  focusing  our  attention  on 
requirements  for  Maverick  training  aids  within  the  Navy’s  A-6E  community.  Strategies  for 
implementing  EO  simulation  on  micro-computers  for  NCTS  personnel,  training  officers  and 
A-6E  aircrews  are  influenced  by  a  combination  of  aircrew  desires  that  EO  training  devices 
contain  courseware  for  tactical  decision-making. 

To  meet  this  requirement  economically,  we  plan  to  incorporate  target  lists  and 
environmental  databases  developed  by  the  armed  services  and  implemented  in  the  MKIII 
EOTDA  developed  by  the  Air  Force.  In  addition,  we  will  examine  operational  data  to 
establish  bounds  on  laser  and  infrared  designation  and  acquisition  performance.  We  plan  to 
use  data  obteined  in  the  Persian  Gulf  War  to  supplement  IR  data  we  have  in-house  to 
establish  oj^rational  bounds.  If  upon  comparing  these  results  to  MKIII  EOTDA  data  we  find 
a  large  disparity  in  the  results,  our  experience  has  taught  us  to  rely  heavy  on  the  operational 
data.  This  is  especially  true  if  the  ultimate  training  device  is  to  be  validated  for  use  in 
mission  planning  and  tactical  decision-making.  The  following  database  comparisons  illustrate 
the  fXJtential  improvement  in  META  simulation  planned  for  Phase  IV. 


PHASE  IV  Database  Forecasts 

SLIDE  5® 

META 

MKIII  EOTDA 

META 

Current 

Available 

Proiected  fPHASE  IV) 

Number  of  IR  Targets  2 

24 

10 

Number  of  Laser  Targets  2 

43 

20 

Number  of  IR  Backgrounds  0 

38 

20 

Number  of  Laser  Backgrounds  2 

169 

20 

With  the  MKIII  Electro-Optical  Tactical  Decision  Aid  (MKIII  EOTDA)  now 

available,  we  have  access  to  a  number  of  submodels  developed  by  the  DOD  to  simulate  laser 


and  infrared  acquisition.  It  should  be  noted  that  MKIII  EOTDA  cannot  run  under  META  as 
a  subroutine.  Although  we  do  have  source  code  for  LOWTRAN,  the  cost  required  to  adapt 
LOWTRAN  to  META  is  beyond  the  scope  of  our  contract.  These  EO  submodels  are 


identified  below: 
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SLIDE  51 


TMpTTT  PFOTTIRFD  GENERALLY  FOR  LASER  SUBMODELS 

Submodel  Number  of  Input  Paramgiers 

Target  Background  2 

Transmissivity  12 

Sensor  Performance  9 

Designator  Range  2 

SLIDE  52 


TNPTTT  REOTTTRFD  FOR  INFRARED  SUBMODELS 


Submodel 

Insolation 
Sky  Radiation 
Target  Contrast  Model 
Transmittance 
Sensor  Performance 


Number  of  Input  Parameters 

9 

4 

16 

12 

11 


As  observed  from  the  large  number  of  parameters  required  by  the  1^  ^  mfmed 
submodels,  a  large  amount  of  time  would  be  required  to  input  data  into  MKin  EOTDA 
format.  In  keeping  with  our  "Bootstrapping"  approach,  we  have  idenUfied  a  smaller  subset 
of  inputs  to  build  simulation  models  around  A-6E  training  requirements. 


SLIDES  53,  54,  55,  56 


SLIDE  57 


To  summarize  the  important  results,  findings  and  lessons  learned  over  the  past  four 
years  of  EO  "Bootstrap"  modelling,  we  offer  the  following. 

®  Over  twenty  years  of  progress  made  by  the  DOD  in  EO  research  and 

mcdelling  have  been  incorporated  into  Maverick  PC-based  training  devices  as 

a  result  of  the  META  program. 

•  "Bootstrapping"  methods  have  worked  just  fine  and  have  kept  overall  training 
system  costs  down. 

To  training  system  developers,  we  offer  the  following  tenets  of  good  "Bootsmpping" 


SLIDE  58 

-®  Accept  little  data  over  no  data 

-®Do  not  over  invest  in  rigor  and  sophistication 

-®Temper  patience  with  progress 

-®KISS  (Keep  it  simple)  Always 
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Slide  1 
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Slide  2 


EO  MAVERICK  MISSILE  FAMILY _ — 

-  guidance  display 

guidance  designation  technology  presentation 
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LA^R  DESIGNATOR  AND  LASER-RjIDED  WEAPOl^ 
-MAVERICK  Missile  (TV\Infrared\Laser) 
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EOTDA 


EO  MODELLING  FOR  META 
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Slide  10 


ANALYTICAL  TOOLS 
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METEOROLOGICAL  RANGE 
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METEOROLOGICAL  RANGE 


236 


Slide  34  (Continued) 
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designation  energy 
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TRANSMISSION  TO  TARGET 
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ENERGY  REFLECTED  BACK  TO  DESIGNATOR 
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AREA  OF  ENERGY  CONCENTRATION 
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Slide  43 


ENERGY  CONCENTRATION  AT  SEEKER 
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DETERMINE  IF  ACQUISITION  OCCURS 
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F  LOWTRAN  AND 


SUMMARY  OF  PHASE  III  LESSONS  LEARNED 
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PHASE  IV  DATABASE  FORECASTS 
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INPUT  required  for  LASER  SUBMODELS 


253 


Slide  51 


IgTASE  xV  METAgMPUTS 
Infrared  Simulation 
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GOOD  BOOTSTRAPPING  TENETS 
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Simulation  and  Dynamic  Te=hn?i«’'°  standardised 

Modelling  Techniques 


Christos  Pandelaras 
Robert  E  Palmer 

AERO  ANALYSIS  DIVISION 
NAVAL  AIR  DEVELOPMENT  CENTER 
WARMINSTER,  PENNSYLVANIA  18974- 


ABSTRACT 

The  Navy  has  "g^^^^se'^capabilit^^^^ 

capabilities  for  tLt  results,  as  well  as  the 

analysis  and  evaluation  cteristics  for  ensuring  safe 

prediction  of  vehicle  i  ^l^tions,  these  models  provide 

flights,  in  supporting  real-time  simulati^^^^^^^^  analyses 

the  capability  to  rapidly  p  cj-andardized  component  modelling 

and  sensitivity  studies  using  used  for  this 

techniques.  The  ®n9ineering  Y  P  ^^j^j^able  from  Boeing 

modelling  is  EASVS,  which  «  \o  utilise  a  generic 

Computer  Services.  nciina  ore-defined  components  such  as 

model  which  was  function  generators,  and  switches  to 

lags,  lead-lags,  integrators ,func^  and  control  system 

represent  the  vehicle  s  aerodynami  ,  p  P  which  the 

characteristics.  This  p*re-defined  components  and/or 

models  are  developed,  using  ®i^®  P  ^  f  modifying  control 

user  defined  FORTRAN  components,  ^  c)  the  types  of 

laws  or  performing  sensitivity  •  ^h-tion  transfer  functions, 

analyses  available  (steady  state  s^^ulat ion  t^^^^ 

root  locus,  etc).  its  achieved  by  using  these  modelling 

demonstrate  the  use  and  P®neli  ™i_time  simulation  for  analyzing 
techniques  in  conDunctxon  case,  without  any 

US  Navy  targets  will  Xj^ed  pitch  and  roll  oscillations 

control  inputs,  flight  ®  ,  mode  was  turned  on  and  off.  This 

when  the  altitude  hold  or  some  other  non-linear 

effect  was  indicative  of  ®  real-time  simulation  model 

dynamic  characteristic.  expensive  and  time  consuming  to 

developed  by  the  ,  expensive  method  was  needed  to 

change,  another  quick  ^  existing  model  of  a  similar 

verify  the  hysteresis  hysteresis  effects  in  the 

vehicle  was  test^conditions  were  then  simulated  and 

actuator  model.  J^®  ^  hvsteresis  characteristics  were  examined, 
the  sensitivity  to  A_racteristics  as  the  flight  test  in 

The  results  exhibited  damping  of  the  oscillations  for  a 

both  the  frequency  and  ^  ^  ^  The  contractor  subsequently 

representative  rtraTt^a^rsAinKages  and  verified  the 

SKU?™ce  ol  fhysteresis  which  he  has  since  ccrrected. 
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SiiBtalatioB  and 


AnmlYsss  of  Targets  Using  Standi 


by 

Christos  Pandelaras 
Robert  E  Palmer 

AERO  ANALYSIS  DIVISION 
NAVAL  AIR  DEVELOPMENT  CENTER 
WARMINSTER,  PENNSYLVANIA  18974-5000 


T  MTRODUCT I ON 

current  U.S.  Havy  targets  have  become  very  expensive  to 

S "u'nsafTfUght  p/ior  to  flight 

can  ^^"sfgSiicJnt?^  ^^00/  by  fiUzing  these 

prt«teT%n°  n4"  tf „Sr  “mlny  J'^5nrH\"o4uppt'rt  b"oth 

■;  1 1  ne-t-t-arfic;  the  various  moaeis  typicaixy 

Illustrates  tnevario  Degree  of  Freedom  (DOF)  longitudinal,  3 
DOF^lateral/directional,  and  finally  a  full  6  DOF 

“Li"  provides  an  tiYtfa^es^^tirsemodeL  arL  bi^ltL^ 

°SSSdlsLky%»^^ 

over  the  past  ten  rcSsluUy""  mod"ellLg 

pLIied'airc^LYS"^  » 

6  DOF  model  of  the  vehicle  from  'eLy?  (Sting  Computer  services, 

,TT  TL“wo%tLtt^  rLLsLLsTo/tlartiL  with  the  most 
reference  (1)).  nonlinear  6  DOF  model  will  eventually 

complex  case  first  are,  provides  the  capability  to 

have  to  be  3  Lf 

extract  the  3  DOF  ^  D°F  amount  of  time 

With  virtually  no  effort.  Therefore  a  sig  developed.  Figure  2 
can  be  saved  since  only  one  model  has  to  me  aeveiop 
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illustrates  this  aircraft  such  as  the  A-6,  T-45, 

support  flight  dynamics  analyses  of  airc  approach  is  that 

F-K,  and  others.  Another  benefit  of  us^ijig^  analyses  capabilities 
EASY5  also  contains  a  ^  s  ^root  locus  diagrams,  and  the 

such  as  functions  which  can  quickly  and 

hl^ppUe-d^”  specific  prohlen  investiqations. 

During  the  past  two  years  ?his"  liig 

support  of  our  targets  P I  including  the  BQM-34S, 

technique  to  a  number  of  one  case,  we  identified 

BQM-74C,  and  BQM-74E  with  prevented  the  vehicle  from 

a  hysteresis  effect  in  an  we  have  used  our 

safely  performing  its  law  and  engine  modifications  to 

models  to  examine  potential  control  1  g  flight 

the  BQM-34S  to  determine  what  ^^^sting.  We  are  convinced 

dynamic  characteristics  pr^or  ^  ^  provides  the  Navy  with 

that  this  standardized  the  lowest  cost.  In 

the  nost  effective  analytical  capabili^ 

ISnirr^Inrtt  Se  tre^eTcdtis  ^?n"  analysing  the  two  example 
cases  mentioned  above. 


model  DEVELOPHENT 

The  modelling  of  a  aynas^o  eyeten  in_^EhSV5  il^e^ieved^using 

Son^o°n"eitI..  an^ITsfr' defined  -dales  caUea  ..p™ 

ser?e  as  system  deecriptive  building  blocks_.^^A  mo^^  P  ^ 

depicts  either:  a)  ®  •  tjie  longitudinal  axis,  b)  a 

calculation  of  forces  and  sinale  pole  switch  that  can  be 

hardware  function  which  ^imulat  gl  surface,  or  c) 

used  to  describe  a  step  or  pulse  inpur  o^r^  ^  summer,  lead-lag,  or 

standard  types  of^  components  "Standard"  and 

transfer  function.  ^  outputs.  Whether  you  describe 

••FORTRAN"  have  specific  inputs  and^^^tp  .'FORTRAN" 

your  system  using  3^®^ .  v,ot-h  the  inputs  and  outputs  of 

LmponeAts,  or  a  <=°"?i"tl'°",fer?v  conne;teno  insure  that  the 
each  component  need  to  be  PfoP  J„ebraic  loops.  When  putting 
desired  system  has  or  f^ 

together  a  JJ°^®Ls\hat  need  to  be  described  and  connected 

vehicle,  the  “^1°^  motions  are  shown  in  Figure  3. 

in  order  to  correctly  qi^en  target  and  are;  a) 

These  elements  are  ^  system  tables,  b)  aerodynamic 

aerodynamic,  engine,  f^rce  and  moment  buildups,  c) 

fo^poieSs  can  use"  in  bhe  development  of  these 
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of-freedom  bare  airframe  nonlinear  dynamics  of  any  aircrart  or 
target o  Figure  4  shows  the  schematic  diagram ^of  this  bare  airframe 
model  using  standard  EASY5  components,  including  various  inputs  and 
outputs  o  The  four  standard  components  used  are  AV,  LO,  LD,  and  SD 
which  are  described  below  along  with  other  commonly  used 
components  o 


_ proponents  LO  and  LD  combine  the  eternal  forces  ana  momenrs 

with  the  aerodynamic  forces  and  moments  in  the  body  axis  and  output 


we  JOTOWy  All'll 

control  surfaces  that  hard_^  - 

To  de^l  with  these  types  of  vehicles^  the  utilization  of  the  user 
defined  FORTRAN  components  becomes  beneficial.  Using  FORTRAN 
components  for  computation  of  the  forces  and  moments  or  for  any 
other  vehicle  function,  allows  the  user  to  make  the  model  specific 
to  the  vehicle.  Creating  a  FORTRAN  component  is  very  simple.  Just 
as  each  standard  component  has  a  name  (AV,I^,LD,SD) ,  so  does  each 
FORTRAN  component.  All  FORTRAN  component  names  must  start  with  the 
letters  s^FO”  as  shown  in  Figure  5.  The  next  two  characters  in  the 
name,  for  both  standard  and  FORTRAN  components,  are  used  to 
distinguish  between  multiple  occurrences  of  the  same  component  in 
the  model.  The  ''LOCATION-"  indicates  the  start  of  a  new  component, 
"FOAT"  indicates  that  this  is  a  FORTRAN  component,  while  "INPUTS='', 
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;H?SS  for::",  ani  co^anas  used  the  reader  rs 

•  air  vehicle  standard 
viave  discussed  the  four  °pe.nmty^  components  in 
So  far  we  ha  use  of  F  _j;,rd  components  and 

components  ^^^'^/^^i-here  are  dozens  ^^'JjSiher  aslist  in  the 

developing  a  to  the  user  standard  components, 

subroutines  l^udel .  Not  all  of  these  standara^ 

development  of  .^^j^uiques  will  he  discus  One  of  these 

subroutines,  an  be  presented  tot  the  ^  used  in  the 

frequently  used  will  bej^  ^Ae  next  section. 

This  IS  discussed  rn  th 

H  FV,  «.d  FW  Table  Look-up  subroutines  ^ 

Most  air  vehicle  m<^els  usu^aUy^«^^  SedW* ’conduefi 

tables  .Seduli  tables)  ,  f^at  §r^«ded_^„_^^, 

control  system  gain  user  with  FORT^  look-ups. 

simulation.  EASY5  P  components  to  perfo  .  the  ''FV 

Jan  be  used  within  FORTR^oomF^^^^^^^^^g^gual  the  "FW” 

The  ''FU«  subrout:^eper^^^_^^^^^^^  3^  look-up 

subroutine  P®tt<^  ^  three-dimensional  done  through  the 

subroutine  performs  „.;ne  dimensional  and  i  available  that 

ISat  can  be  accessed  is  nine^di^  'th^s  the  call  sequence 

Z  P?tibm  the  same^iunctions.^^^  „,,,„as. 

that  IS  neede  ,-vstem  description  is 

r^e^lnp^^s  and^^^ 

model.  It  Jf^®i\i^that  can  be  reused  for  words,  one  model 

rnrL\“i^e%r«ithU^^ 

of  a  dynamic  system 
discussed  above. 


„narated  EASY5  allows  the 

once  an  fUe^leSuesting  the  analyses^ 

iser  to  «eat®  an  an  ^  y,^^  ?"alyses,  th^^^P^  values  for 

sssSiC“Er..;r‘.».-” 

parameters  (constants) 
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also  accomplished  in  the  analysis  file.  A  sample  analysis  file 
recjuesting  a  steady  state  calculation  and  a  simulation  is  shown  in 
Fiaure  8.  In  this  example,  the  analysis  file  consists  of:  a)  a 
title  description  (’®TITLE=EXAMPLE  ANALYSIS  FILE”)  ,  b)  a  command  to 
define  parameters  C”PARAMETER  VALUES”),  c)  a  command  to  set  up 
table  values  ("TABLE”)  to  define  the  control  input,  d)  definition 
of  initial  state  values  ("INITIAL  CONDITIONS”)  ,  e)  determination  of 
trim  condition  ("STEADY  STATE”),  and  f)  the  command  to  perform  a 
simulation  ("SIMULATE”).  The  parameter  values  and  table  coi^ands 
are  used  to  define  all  of  the  constants  and  table  data  specified  in 
the  model  description.  The  initial  conditions  command  is  used  to 
initialize  all  or  some  of  the  state  variables  in  the  model. 
Discussion  of  the  steady  state  and  simulate  analysis,  along  with 
several  other  type  of  analysis  is  presented  below. 


Steady  Stat®  Analysis 

The  steady  state  analysis  is  used  to  find  an  eguilibrium 
(trim)  point  for  the  model.  This  point  occurs  when  the 
root“m®an“sguare  value  of  all  active  system  rates  is  near  zero.  The 
method  used  is  a  Newton-Raphson  iterative  approach,  using  knowledge 
of  the  system  Jacobian,  that  algebraically  manipulates  the  system 
states  until  the  algorithm  converges  to  a  solution.  This  iterative 
process  is  repeated  until  either  the  norm  of  the  rates  becomes  less 
than  0.0001  or  the  maximum  number  of  iterations  (e.g.  in  Figure  8 
"SS  ITERATIONS=40")  has  been  reached.  If,  during  the  steady  state 
analysis,  a  state  causes  the  Jacobian  matrix  to  become  singular, 
that  state  will  be  frozen  at  its  current  value  and  the  solution 
process  will  continue.  In  the  example  analysis  file  in  Figure  8, 
the  steady  state  command  is  solving  for  a  trim  solution  of  an  air 
vehicle  at  a  specified  flight  condition  and  configuration.  Once 
this  trim  solution  is  reached,  that  operating  point  becomes  the 
starting  point  for  the  simulation  that  follows  by  using  the  "XIC“X 
command. 


Siffiulatioffi  Analysis 

simulation  analysis  is  the  process  of  numerical  integration  of 
the  model's  equations  of  motion  through  time.  It  is  used  to 
generate  time  histories  of  the  system  outputs  to  evaluate  the 
model's  behavior.  As  shown  in  Table  1,  there  are  a  total  of  nine 
numerical  integration  techniques  available  in  EASY5. 
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Marne 

Method 

Order 

Type 

BCS  Gear 

BCS  Modified 

Stiff  Gear 

Variable 

Variable- 

Step/Implicit 

Runge-Kutta 

Variable-Step 

Variable- 

Step/Explicit 

Runge-Kutta 

Fixed-Step 

4  th 

Fixed- 

Step/Explicit 

Fixed-Step  Huen 
Method 

2nd 

Fixed- 

Step/Implicit 

Euler 

Fixed-Step  Euler 

1st 

Fixed- 

Step/Explicit 

Adams 

Adams-Bashforth 
Predictor  Adams- 
Moulton  Corrector 

2nd-12th 

Variable- 

Step/Explicit 

Stiff  Gear 

Hindmarsh  Version 
of  Gear  Algorithm 

variable 

Variable- 

Step/Implicit 

Linear 

Simulation 

Linear  Simulation 

N/A 

N/A 

User-Defined 

N/A 

N/A 

N/A 

TABLE  1:  Integration  Methods 


In  the  example  analysis  file  in  Figure  8,  the  i”t®9rat ion  method 

Loser,  was  Runge-Kutta.  There  are  InTtSey 

associated  with  the  simulation  analysis  shown  in  Figure  8,  ana  rney 

are  defined  below; 

TINC  -  is  the  integrator  step  size. 

TMAX  -  is  the  maximum  value  of  simulated  time. 

OUTRATE  -  is  an  integer  number  that  is  multiplied  by  the  value 
OHTRATE  a^n^^in  ^  g  „cord  simulation 

PRATE  -  defines  how  often  you  want  output  data  printed. 

Figure  9  is  an  example  of  a  time  history  plot  generated  from  the 
sample  analysis  file. 


Transfer  Function  Analysis 


The  Transfer  Function  analysis  is  used  to  investigate  the 
stability  performance,  and  frequency  related  response  of  the 
Sei  ■nJa  Inliysis  oonsists  of  two  parts.  First,  the  creation  of 
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a  linear  model  between  two  points  in  the  system  about  the  current 
operating  point,  followed  by  the  calculation  of  the  poles,  the 
zeros,  and  the  leading  coefficient  of  the  transfer  function.  Once 
these  have  been  calculated,  a  frequency  response  analysis  can  be 
performed  between  the  two  specified  points  of  the  system.  The 
frequency  response  function  computed  is  the  steady-state  response 
of  the  linearized  system  to  a  sinusoidal  input.  This  frequency 
response  is  calculated  by  evaluating  the  function  over  a  range  of 
frequencies  which  is  either  input  manually  or  determined 
automatically.  Three  formats  are  available  within  EASY5  for 
plotting  the  results  of  a  frequency  response  calculation.  They 
are?  Bode  plot,  Nichols  chart,  and  Nyquist  diagram.  Figures  10, 
11,  and  12  are  representative  plots  of  each  of  these  formats. 


The  Root  Locus  analysis  calculates  the  loci  of  the  system 
eigenvalues  as  a  function  of  a  specified  parameter.  The  analysis 
can  be  performed  as  a  function  of  any  operating  point  value,  or  any 
system  parameter.  Since  the  analysis  can  be  applied  to  both 
continuous  and  sampled  data  systems,  the  output  can  be  given  in 
either  the  s-plane  or  the  z-plane.  Before  executing  this  analysis, 
the  user  needs  to  specify  in  the  analysis  file  which  quantity  of 
the  model  to  vary,  through  what  range,  and  with  what  resolution. 
The  commands  used  to  do  this  are  listed  below; 

RL  PARAMETER  -  Root  locus  parameter  name. 

RL  START  “  Initial  value  of  root  locus  parameter, 

RL  STOP  -  Final  value  of  root  locus  parameter. 

RL  POINTS  “  Number  of  points  that  make  up  root  locus. 

An  example  of  a  root  locus  plot  that  can  be  generated  in  EASY5  is 
shown  in  Figure  13  where  the  numbers  inside  the  plot  represent  the 
value  of  the  RL  PARAMETER, 


The  Eigenvalue  Sensitivity  is  another  commonly  used  analysis 
which  measures  the  sensitivity  of  the  system  eigenvalues  to  a 
change  in  a  specified  system  parameter.  The  eigenvalue  sensitivity 
measure  is  the  ratio  of  the  percentage  change  of  each  eigenvalue  to 
the  percentage  change  of  the  parameter  for  which  the  sensitivity  is 
to  be  measured.  To  specify  the  eigen  parameter  in  the  analysis 
file,  the  following  command  is  used; 

EIGEN  PARAMETER  -  Eigen  parameter  name. 

To  execute  an  eigenvalue  sensitivity  analysis,  the  following 
command  is  used. 
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EIGEN  SENSITIVITY 


The  analysis  capabilities  presented  in  this  section  are  only 
a  portion  of  those  available  within  the  EASYS  program.  In  the  next 
section,  two  example  cases  will  be  presented  that  apply  some  of 
these  analysis  capabilities  to  actual  target  vehicles.  This  was 
done  in  order  to  evaluate  their  dynamic  response,  or  to  predict 
their  sensitivity  to  potential  modifications. 


APPLICATION  CASES 

As  mentioned  in  previous  sections  of  this  paper ,  EASYS  dynamic 
analyses  models  of  various  targets  have  been,  and  are  currently 
being  developed  at  the  Naval  Air  Development  Center.  These  models 
utilize  the  approach  of  first  developing  a  full  6  DOF  nonlinear 
representation  of  the  vehicle  and  then  exercising  the  inherent 
capabilities  of  EASYS  to  extract  linear  models,  or  to  perfora  a 
variety  of  dynamic  analyses.  This  section  of  the  paper  describes 
in  more  detail  two  recent  applications  of  this  approach.  For  these 
two  examples,  EASYS  models  were  used  successfully  to  both  evaluate 
and  predict  the  dynamic  response  characteristics  of  target 
vehicles.  In  one  case,  it  resulted  in  identifying  a  deficiency  in 
the  contractor  design  which  was  subsequently  corrected  at  minimum 
cost  to  the  Navy.  In  the  second  case,  it  is  allowing  the  Navy  to 
investigate  potential  changes  to  an  existing  vehicle  prior  to 
flight  testing  to  determine  their  effectiveness.  This  will  result 
in  more  efficient  planning  and  conduct  of  flight  tests  of  the 
vehicle  when  the  modifications  are  implemented.  These  two  examples 
are  presented  below. 


Example  ^1: 

In  recent  flight  tests  of  an  experimental  target,  whose  dynamic 
characteristics  are  similar  to  those  of  a  BQM-74C,  unacceptable 
pitch  and  roll  oscillations  were  observed.  These  were  severe  enough 
to  prevent  the  vehicle  from  meeting  its  mission  performance 
objectives.  The  oscillations  occurred  when  the  flight  control 
system  logic  commanded  the  vehicle  to  enter  an  altitude  hold  mode. 
Immediately  after  this  command  was  given,  the  vehicle  started  to 
pitch  and  roll  as  shown  by  Figures  14  and  15.  The  data  presented 
in  these  figures  have  been  extracted  from  actual  stripcharts  and 
were  selected  to  illustrate  the  relative  frequencies  and  amplitudes 
of  these  oscillations.  As  shown,  both  frequencies  were 
approximately  0.75  cycles/ second  (cps)  while  the  amplitude  for 
pitch  was  5.0  degrees  and  the  amplitude  for  roll  was  7.5  degrees. 
After  examining  these  data  in  more  detail,  it  appeared  that  a 
possible  cause  of  these  oscillations  was  the  presence  of  nonlinear 
actuator  dynamics  typical  of  a  hysteresis  effect.  In  order  to 
quickly  determine  whether  this  assessment  was  correct,  an  existing 
EASY5  model  of  the  BQM-74C  was  modified  within  two  days  to  include 
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a  hysteresis  component  in  the  same  control  path  as  the  actuator. 
This  approach  is  shown  schematically  in  Figure  16.  Representative 
values  of  the  hysteresis  characteristics  were  then  determined  based 
on  previous  experience  with  other  vehicles,  and  analysis  conditions 
were  set  up  to  closely  match  the  speed  and  altitude  of  the  flight 
test  data»  A  number  of  runs  were  then  made  to  determine  if  the 
introduction  of  the  hysteresis  component  would  induce  pitch  and 
roll  oscillations  similar  to  those  observed  in  the  flight  tests. 
Figures  17  (pitch)  and  18  (roll)  show  the  results  of  this  analysis 
were  very  successful.  In  both  these  figures  the  baseline  (no 
hysteresis)  response  of  the  BQM“74C,  following  a  control  input 
disturbance  at  4.0  seconds,  is  shown  for  reference  as  a  dashed 
line.  The  solid  curve  shows  the  response  with  the  representative 
hysteresis  effect.  Both  the  pitch  and  roll  responses  closely 
matched  the  frequency  and  amplitude  characteristics  of  the  flight 
test  data  and  supported  the  hypothesis  that  the  actuators  had  some 
hysteresis.  Prior  to  additional  flight  tests,  the  contractor's 
analyses  were  examined  in  more  detail  and  it  was  determined  that 
only  linear  actuator  characteristics  had  been  modelled.  Based  on 
a  linear  analysis,  it  was  not  surprising  that  the  contractor  had 
not  predicted  a  hysteresis  effect.  Subsequently,  the  contractor 
conducted  bench  tests  on  the  actuator  to  measure  its  dynamic 
characteristics  and  verified  that  hysteresis  was  present. 
Modifications  to  the  actuator  were  then  made  and  follow-on  flight 
tests  showed  a  significant  improvement  in  the  vehicle's  pitch  and 
roll  responses  in  the  altitude  hold  mode. 


Example  #2 

The  Navy  has  recently  been  investigating  the  impact  of 
updating  the  flight  control  system  in  its  BQM34S  target  from  an 
analog  system  to  a  digital  system,  similar  to  the  one  that  is 
currently  in  the  Air  Force's  version  of  the  target.  The  Naval  Air 
Development  Center  has  developed  a  dynamic  simulation  model  of  the 
BQM34S  that  incorporates  this  new  flight  control  system.  Some 
preliminary  results  are  presented  that  demonstrate  the  type  of 
predictive  dynamic  response  analyses  which  can  be  performed  with 
EASY5  before  proceeding  to  flight  testing.  Two  analysis  were 
performed 2  a)  a  trim  solution  of  the  target  was  obtained  at  a 
nominal  flight  condition,  and  b)  a  longitudinal  control  input  was 
applied  to  the  bare  airframe  model.  The  flight  condition  used  for 
this  analysis  wass 

Mach  =  0.6 

Altitude  =  10,000  feet 

Weight  =  2100  pounds 

Center  of  Gravity  (C.G.)  =  25%  Mean  Aerodynamic  Chord(MAC) 

Using  the  "STEADY  STATE"  command  the  target  was  trimmed  at  this 
flight  condition.  The  trim  solution  showed  the  target  trimming  to 
an  angle-of-attack  (ALPHA)  of  2.817  degrees  (Figure  19a),  with  a 
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FIGURE  3%  The  major  elements  associated  with  the  development  of  an 
air  vehicle  dynamic  model. 


FIGURE  4s  Bare  airframe  schematic  diagram  using  air  vehicle 
standard  components. 
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u  c:n  EA  SD,DEF2AIL 

,  oKQ  eoAT  INPUTS=QC  AV,W  SD,t.A  , 

iCATION=259  FOAl 

add  VARIABLES=AKP , RADWl , RADEAl 

fortran  statements 

*  TP  roc  AV  .LTo  100.)  AKP  =J;;° 

QC  AV  .LT.  10°-> 

ACT  =  96.64/QC  AV  +  0.0334 

500  RADWl  =  W  3 

RADEAl  =  EA  SD(l)/57.3 

f  »  fortran  component  as  defined  in  batch 
FIGORE  S:  Example  of  a  FORTRAN  c 

model . 


CALL  FU(tn,dv,idvl,anl) 

CALL  FV(tn,av,idvl,iav2,anl,an2) 

CALL  FW  ( tn ,  av ,  iavl ,  iav2 ,  iav3 ,  .m ,  an2 ,  «>3 ) 


where ; 

tn  is  the  l°f"JP,^^e^tSe-dependent  variable) 
dv  is  the  fij-st-table  independent  variable) 

idvl  is  an  /  second-table  independent 

idv2  is  an  input  (the  seco»  .  ,  , 

variable)  r-n-table  independent  variable) 

idv3  is  an  input  _  for  idvl  (any  negative 

is  an  e*Wapolation  flag  for 

real  value  "i“  P”''CTaa  for  iav2  (any  negative 
Bn2  is  an  =’‘““P?}?’^^°,„ent^exttapolation) 

real  value  for  iav3  (any  negative 

an3  is  an  'Siifp«vlnl®ex?rapoIatiL) 

S.  cairseguences  and  argument  definitions  for  table 
look”Up  subroutines. 
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Figwr©  7i  a)  Block  diagram  of  air  vehicle  control  systemo 
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Figure 


in  EASV5. 
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TITLE=EXAMPLE  ANALYSIS  FILE 
PARAMETER  VALUES 

S=179»64  CBAR=5.83  SPAN=30o8 

GW=11500o0  FSREF=258.47  FSCG=254.972 

WLREF=28o46  WLCG-27.61  FLAP=50o0 

SLAT=15. 0 

TABLE,  A2TTBFP,  8 

0»0,  0.99,  1.0,  1,01,  3,0,  3,01,  5,0,  10,0 
0,0,  0,0,  15,0,  15,0,  15,0,  0.0,  0.0,  0,0 
& 

INITIAL  CONDITIONS 

U  SDC1)=222,6  U  SD(2)=0,  U  SD(3)=11.666 

W  SD(1)=0,  W  SD(2)=0,  W  SD(3)=0, 

EA  SD(1)=0.  EA  SD(2)^3o0  EA  SD(3)=0, 

ALTSD=500,0 

’k 

SS  ITERATIONS=40 
STEADY  STATE 

it 

XIC-X 

k 

INT  MODE  =  RUNGE  KUTTA 

TINC=0,01,  TMAX=10,0,  OUTRATE=l,  PRATE=1 

SIMULATE 


FIGTOE  ®i  Example  of  an  analysis  file  in  EASY5  batch  mode. 
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Figure  11:  Example  of  a  Nichols  chart. 
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Roll  Angle  (degrees) 


Fig«re  IS: '  Flight  Test  Data  Results  -  Roll  Respond. 
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Hysteresis 


Figure  16:  Modified  BQM-74C  EASY5  Dynaeio  Model 
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Pitch  Angle  (degrees) 


Frequency  =  1  .02  cps 
Amplitude's. 7  degrees 


Velocity=330  KIAS 
Altitude=1  0,000  feet 


Figure  17;  EASY5  Tlesults  for  Pitch  Respoms®, 
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Frequency=0.75  cps  Velocity— 330  KIAS 

Amplitude=3.5  -  5.0  degrees  Altitude  =1  0,000  feet 


Figure  18:  EASY5  Results  for  Roll  Response. 


Figure  19:  a)  Trim  solution  at  nominal  flight  condition. 


re  19:  b)  Trim  solution  at  nominal  flight  condition. 
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A  tliqht  simulator  normally  uses  components  that  are  specific 
for  the  qiven  simulated  vehicle.  For  example,  it  the  simulator  was 
designed  to  simulate  the  environment  of  a  F/A-18,  one  could  not  use 
the  simulator  for  A-6  simulation.  However,  by  changing 
simulation  software  and  One 

equipment  could  be  used  to  simulate  any  ,  ^coctbit  shell 

could  also  just  change  the  interior  components  of  a  cockpit  shell 

to  simulate  a  family  of  aircraft.  This 
would  allow  simulation  and  training  for  a 

platforms  in  a  little  more  space  that  a  single  simulator  tak  . 

A  reconfigurable  cockpit  would  increase  the  utilization 
capability  of  simulators.  However,  because  of  the  shortage  of 
available  simulators,  simulator  use  is  usually  24  hours  a  day  and 
an  added  aircraft  simulation  capability  to  a  platform 
^alue.  A  reconfigurable  cockpit  with  built-in  computational 
hardware  would  have  value  if  it  could  be  deployed  to  sites  wh 
siLlators  are  not  available.  These  candidate  sites  would  most 
likely  have  a  small  number  of  users  for  a  given  airframe  making  it 
impractical  to  have  a  single  aircraft  simulator  stationed.  A 

reconfigurable  cockpit  would  allow  for  multiple  P  Yi^^notentiar 
increasing  utilization  of  the  simulation  assets  to  full  potenti  . 
A  deployable  reconfigurable  cockpit  would  make  available  simulation 
in  areas  that  are  space  restricted  and/or  temporary  stations. 

With  the  recent  advances  in  technology,  the  cost  of  a 
simulator  ten  years  ago  can  be  built  today  for  a 

previous  cost.  Improvements  in  technology  have  made  available 
greatly  increased  fidelity  and  simulator  capability.  Technology 
Lw  allows  for  a  high  fidelity  simulation  environment  at  a  reduced 
cost.  To  apply  the  latest  in  reduced  cost  simulation  technology  to 
the  deployable,  reconfigurable  cockpit  would  increase  the  access  to 
simulators  by  allowing  multiple  procurements  of  the  reconfigurable 
Cockpits  for  the  price  of  one.  Of  course  the  low  cost  systems 
would  not  have  all  the  features  as  the  current  fixed  site 
simulators  but  would  have  partial  capability  at  least  to  maintain 
pilot  skills  while  away  from  conventional  training  sites. 


Reconfigurable  Cockpit  Alternatives 

As  with  any  classical  System  Engineering  design,  a  top-level 
trade-off  analysis  of  approaches  was  conducted.  This  analysis  was 
conducted  early  during  the  Phase  I  effort,  m  order  to  establish  a 
clear  direction  for  all  subsequent  design  activities  to  proceed. 
SYMVIONICS  addressed  three  fundamental  approaches  to  proyi  ing 
variety  of  cockpit  simulations  (i.e.,  different  cockpits),  which 
could  be  used  for  tactics  team  training.  These  approaches  were  as 

follows : 


Option  1:  Electronic  replication  (and  reconfiguration)  of 
crew  system  Controls  and  Displays 

Option  2;  Interchangeable  cockpits  (in  their  entirety) 
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•  Option  3:  Interchangeable  individual  cockpit  Control  and 
Display  modules 

The  various  reconf igurable  cockpit  options  were  evaluated  or 
scored  as  alternatives  against  a  set  of  weighted  criteria,  based  on 
a  formalized,  classical,  system  engineering  trade  study 
methodology.  Classical  trade  study  analysis  establishes  concise 
alternatives  to  a  problem,  which  are  evaluated  against  concise 
criteria;  based  on  relative,  established  (apriori)  procedures. 
Alternatives  were  scored  based  on  the  "goodness"  of  the  alternative 
Criteria  used  in  the  evaluation  included:  Fidelity,  cost,  ease  of 
reconfigurability,  logistics  (storage  space  and  number  of  people  to 
reconfigure  cockpit),  reliability,  maintainability,  training 
effectiveness  and  customer  preference. 

The  first  approach,  electronic  replication  (and 
reconfiguration)  of  crew  system  Controls  and  Displays,  is  created 
using  the  so-called  Virtual  Cockpit  concept.  With  this  concept, 
the  cockpit  "panel  space"  is  replaced  with  electronic  display 
devices,  such  as  CRTs.  These  displays  then  become  output  devices 
for  high-resolution  graphics  generators.  The  graphics  generators 
create  a  facsimile  or  graphics  rendering  of  the  various  cockpit 
Controls  and  Displays.  The  display  devices  are  then  overlaid  with 
a  touch  sensitive  "screen"  such  that  crew  inputs  (switch  actions) 
are  accomplished  by  touching  the  appropriate  control  element. 
Appropriate  cockpit  display  responses  are  then  reflected  by  changes 
in  the  graphics  displays. 

The  second  approach  examined  was  a  self-contained 
interchangeable  cockpit  (entire  cockpit) ,  representing  the  various 
aircraft,  to  be  interchanged.  This  approach  allows  for  each 
cockpit  to  be  internally  wired  to  its  unique  configuration.  This 
internal  wiring  is  then  interfaced  with  a  centralized  on-board 
cockpit  processor,  such  that  a  vast  amount  of  cockpit  Control  and 
Display  processing  can  be  done  locally  at  the  cockpit,  relieving 
this  burden  from  the  simulator  host  computer  system. 

The  third  approach,  interchangeable  individual  cockpit  modules 
(with  embedded  processors) ,  allows  for  major  sections  of  the 
cockpit,  to  be  physically  relocated  within  a  common  cockpit 
structure.  With  this  approach,  individual  displays,  control 
panels,  side  consoles,  etc.  are  identified  for  each  aircraft 
cockpit  to  be  represented  in  the  training  system.  A  common  cockpit 
base  structure,  for  these  various  aircraft  cockpits,  is  then 
created  in  which  these  removable  and  replaceable  cockpit  modules 
are  placed.  Therefore,  depending  on  the  particular  configuration 
of  the  cockpit  to  be  used  in  the  team  trainer,  various  modules  are 
either  removed,  installed  or  rearranged  in  this  common  cockpit 
structure . 

Table  1  depicts  a  summary  of  the  advantages  and  disadvantages 
of  each  alternative  option,  as  it  relates  to  the  requirements  of 
this  research  activity. 
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METHOD 

ADVANTAGES 

DISADVANTAGES 

ELECTRONIC  REPLICATION  AND 
RECONFIGURATION  CVIRTUAL 
COCKPIT- 1 

®  SHORT  TIME  TO  RECONFIGURE  COCKPIT 
CONTROL  AND  DISPLAY  ARRANGEMENT 

0  RECONFIGURATION  IS  ACCOMPLISHED  BY 
LOADING  NEW  SOFTWARE.  REFLECTING  A 
GRAPHICAL  RENDITION  OF  THE  COCKPIT 
PANEL 

a  ESPECIALLY  USEFUL  DURING  COCKPIT 
CONTROLS  &  DISPLAYS  DESIGN 

®  LOSS  OF  fidelity  PARTICULARLY  WITH 
RESPECT  TO  tactile  FEEL  AND  THE 
representation  of  THE  ELECTRONIC 
HEAD-DOWN  DISPLAYS 
®  POTENTIAL  FOR  A  NEGATIVE  TRANSFER 

OF  TRAINING 

®  COCKPIT  GEOMETRIES  MAY  BE 

VIOLATED.  FURTHER  AUGMENTING  THE 

LOSS  OF  TRAINING  FIDELITY  AND 

CAPABILITY 

®  DISPLAY  LIMITATIONS  MAY  LIMIT  THE 
AMOUNT  OF  COCKPIT  PANEL  SPACE 

WHICH  CAN  BE  DEPICTED  (LIKE  SIDE 

CONSOLESI 

INTERCHANGEABLE  COCKPITS 

OF  ENTIRE  COCKPITS 

a  ENHANCED  REALISM  AND  FIDELITY  OF 

THE  COCKPIT 

a  EACH  COCKPIT  IS  BUILT  TO  THE  EXACT 
SPECIFICATION  OF  THE  ACTUAL  SYSTEM 
a  MOLD  LINES.  PANEL  GEOMETRY  AND 
CONSOLE  CONFIGURATIONS  ARE 
ACCURATELY  REPLICATED 
a  THERE  IS  A  DRAMATIC  REDUCTION  OF 
COMPLICATED  EXTERNAL  INTERFACES  TO 
THE  SIMULATOR  HOST  COMPUTER 

SYSTEM  (NO  internal  COCKPIT  WIRING 

IS  EVER  CHANGEOI 

®  HIGH  COST 

®  MULTIPLE  COCKPITS  REQUIRE  LARGER 

STORAGE  FACILITIES  FOR  THE  COCKPITS 

AND  SOME  SORT  OF  OVERHEAD  CRANE 

DEVICE  FOR  COCKPIT  INSTALLATION 
*  SPARING  REQUIREMENTS  EXAGGERATED 

INTERCHANGEABLE  INDIVIDUAL 
COCKPIT  MODULES  (WITH 
EMBEDDED  PROCESSORS! 

o  COCKPIT  FIDELITY  AND  REALISM 

PRESERVED 

o  MAJOR  SECTIONS  OF  THE  COCKPIT 

COULD  BE  RECONFIGURED  IN  A  VERY 

RAPID  FASHION  BY  A  SINGLE  TECHNICIAN 
a  COMMON  CONNECTIONS  FOR  SIGNAL. 
POWER  AND  COOLING  PROVIDES  FOR 
EFFICIENT  STANDARDIZATION. 

REGARDLESS  OF  THE  SPECIFIC  AIRCRAFT 
CONFIGURATION 

®  DISTRIBUTION  OF  LOCAL  PROCESSING  OR 
-INTELLIGENCE-  IN  MODULES  ALLOW 

THESE  MODULES  TO  BE  EASILY 

CONNECTED  AT  THE  DIGITAL  LEVEL 

®  AN  INTERNAL  ADJUSTABLE  SUPPORT 
STRUCTURE  FOR  ALL  POTENTIAL 

COCKPIT  CONTROLS  AND  DISPLAYS  (FOR 

ALL  AIRCRAFT  CONFIGURATIONS!  MUST 

BE  CREATED 

o  EMBEDDING  PROCESSORS  WITHIN  EACH 
MODULE  MAY  HAVE  PACKAGING 
CONSTRAINTS 

»  ALL  UNIQUE  MODULES  MUST  BE  SPARED 

Table  1.  Alternatives  Summary 


With  the  utilization  of  the  interchangeable  individual  cockpit 
modules  approach,  utilization  of  operationally  faithful  hardware, 
while  adhering  to  proper  cockpit  geometries,  could  be  attained. 
Most  of  the  disadvantages  indicated  above  were  strictly  design 
issues  which  were  solved  during  the  conceptualization  of  the  LCRC. 
The  approach  would  also  provide  the  capability  of  reconfiguration 
to  a  tandem  seat  configuration,  as  well  as  to  rotorcraft.  As  will 
be  described  in  the  following  section  on  design  characteristics, 
utilization  of  embedded  processors  and  graphics  generators,  with 
standardized  interfaces,  provided  the  needed  modularity  to  provide 
functionality  to  the  design. 
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The  displays  are  considered  removable  components,  which 
integrate  with  the  Main  Instrument  Panel  Module  and  the  Video 
Generation  Module.  The  various  electronic  displays,  such  as  the 
Head-Up  Display,  Head-Down  Displays  and  the  Electronic  Warfare 
Displays  are  NOT  shown  in  the  LCRC  Pictorial  Drawing. 

The  LCRC  hardware  design  is  truly  modular  in  three  different 
ways.  It  is  first,  as  described  above,  comprised  of  individual 
modules  which  are  aircraft  type  specific.  Thus,  the  true 
reconfigurability  is  contained  at  this  level.  Second,  one  LCRC  can 
be  directly  connected  to  another,  so  that  simulation  of  two  seat 
tactical  aircraft  can  be  easily  accomplished.  In  this  manner,  the 
forward  LCRC  Base  Module  merely  acts  as  a  "pass  through"  structure 
for  power,  grounding,  signal  and  cooling  networks.  Third,  the 
entire  LCRC  is  just  one  modular  component  of  the  Tactical  Team 
Trainer  System.  As  such,  the  LCRC  is  designed  to  interface  with 
other  major  subsystems  such  as,  but  not  limited  to,  outside  visual 
graphics  generators,  outside  visual  display  systems,  instructor 
operator  stations,  audio  tone  generators,  and  simulation  host 
computers.  To  this  end,  SYMVIONICS  has  realized  the  necessity  of 
incorporating  systems  engineering  early  in  the  design  of  such  a 
complex  device  as  the  Tactical  Team  Trainer  System,  and  has  defined 
the  LCRC  accordingly. 

The  LCRC  was  designed  to  use  embedded  PC-based  computers  and 
peripherals  whenever  possible.  The  price,  performance,  variety  and 
availability  of  peripherals  for  PC-based  computers  have  been 
improving  tremendously  over  time  and  will  continue  to  do  so  in  the 
future.  Basing  the  design  of  the  Cockpit  on  the  PC  provided  the 
capability  to  keep  the  cost  to  the  absolute  minimum.  The  number  of 
Pcs  required  will  probably  be  able  to  be  reduced  as  time 
progresses.  Additionally,  the  use  of  a  common  structural  base 
provided  for  an  easy  and  effective  method  of  using  intelligent 
building  blocks  to  create  a  desired  cockpit  configuration  in  a 
short  period  of  time. 

In  the  design  of  the  LCRC,  low-cost  solutions  were  desired  and 
expected.  However,  these  low-cost  alternatives  would  not  be  deemed 
acceptable  unless  a  "full-fidelity"  capability  was  still 
maintained.  The  primary  use  for  the  cockpit  was  to  be  for  training 
tactics  and  weapon  deployment,  as  well  as  team  training  (i.e., 
internetted  operations) .  Missions  to  be  examined  include  both  Air- 
to-Air  and  Air-to-Ground.  The  designed  LCRC  was  also  to  address 
shipboard  use.  Therefore,  space  and  external  hook-ups  were  to  be 
minimized.  That  is,  logistics  concerns  were  to  be  analyzed,  such 
that  the  resulting  design  would  be  easily  and  fully  supported  with 
minimal  resources. 

Based  on  specified  top-level  requirements  and  internally- 
derived  detailed  requirements,  a  systematic  design  activity  matured 
a  viable  architecture  for  the  LCRC.  This  design  would  satisfy  all 
known  requirements  and  could  be  used  as  an  integral  part  of 
traditional  training  systems  (such  as  tactics  trainers,  operational 
flight  trainers  and  other  part-task  trainers)  or  even  more  current 
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SIMULATOR  NETWORK  BUS 


Figure  3.  Basic  LCRC  Architecture 


maintainability  than  conventional  simulator  cockpits.  In  addition 
(and  very  important) ,  the  modular  design  allows  for  quicker 
reconfiguration  between  aircraft  types.  A  basic  module 
configuration  is  shown  in  Figure  4. 


PCs  as  the  Interface  Computer 

Several  types  of  computer  systems  were  investigated  for  use  in 
the  LCRC.  For  example,  many  single  board  computers,  peripherals 
and  mounting  racks  are  available  for  VME  and  Multibus 
architectures.  These  systems  are  relatively  expensive  and  bulky. 
They  lend  themselves  more  to  designs  that  feature  a  centralized 
computer  system  rather  than  our  federated  design  concept.  PC  based 
computers  are  less  expensive  than  VME  or  Multibus  systems  and 
peripherals  are  available  to  support  all  of  the  LCRC  requirements. 

A  direct  flow-down  hardware  design  requirement  for  the  use  of 
embedded  processors  within  the  LCRC  is  to  dimensionally  house  at 
least  one  card  cage,  PC  motherboard  and  Input/Output  (I/O)  card  in 
every  module  which  processes  I/O.  This  then  produces  derived 
requirements  for  each  module  to  have  appropriate  power,  grounding, 
signal  and  cooling  be  distributed  to  it. 
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Figure  4.  Basic  Module  Configuration 
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systems  and  trainer  style  fore/aft  seating.  The  SYMV  IONICS^,  .a 
provides  an  effective  and  convenient  mechanism  for  convert  . 
single  seat  aircraft  into  a  dual  seat  (tandem)  air  i 
conf igurat ion . 

Reconf iaurable  rncknit  Operations 

Operations  with  the  LCRC  are  designed  to  allow  a  sing^ 
technician  to  perforin  a  checkout  of  the  entire  cockpit  operation 
a  relatively  short  period  of  time.  This  can  be  accomplished  b  ^ 
on  the  modularity  of  the  design,  as  well  as  by  exploiting  -■ 
processing  capability  and  internal  graphics 

inherent  in  the  LCRC.  The  hardware  design  supports  this  conce^, . 
ihrcSgH  IL  use  of  self-aligning  nodules  quick  disconnect 
fasteners  and  minimal  numbers  of  modules  and  connections.  - 

software  design  also  meets  this  requirement  the  use  ■ 

diagnostic  routines,  which  are  limited  in  speed  J 

operator  running  through  them.  It  is  estimated  to  take  oni 
hours  to  fully  assemble  and  initially  checkout  an  LCRC  m  anv 
single  seat  aircraft  configuration.  More  importantly,  SY^ION,...^ 
estimates  that  it  will  only  take  1  hour  to  reconfigure  t.he__L.' 
between  aircraft  types,  once  initially  assembled. 

The  SYMVIONICS  LCRC  has  also  been  designed  to  provide  compl- 
interoperability  and  compatibility  with  other  simulator  syst^  'f 
which  would  normally  be  provided  within  a  complete  training  sysU- ■ 
These  subsystems  include  the  following:  Simulation  k 

Computational  System,  Visual  Display  System,  Sensor  Simulators 
Motion  System,  Sound  System/ Intercom  and  Instructor  Operator 

Station. 


Summary 

The  reconfigurable  cockpit  technology  described  herein  affords 
the  potential  user  a  viable  fidelity  and  cost  driven  design.  The 
application  and  use  of  commercially  available  products  helps  reduc  • 
manufacturing  and  maintenance  costs  and  assures  increase  sys  -n 
reliability.  The  design  strategy  easily  allows  for  application 
other  aircraft  platforms.  The  design  of  the  cockpit,  even  withou 
the  reconfigurability,  allows  for  a  cost  effective  cockpit  solution 
to  many  low  cost  flight  simulation  requirements.  Thr 

reconfigurability  and  deployable  features  of  the  design  cou  ] 
afford  the  user  a  high  fidelity  simulation  solutions  at  locatio,..- 
almost  anywhere  desired. 

SYMVIONICS  has  developed  the  complete  architecture  and  t 
level  design  for  a  modularized,  reconfigurable  cockpit 
deployable  aircrew  tactics  team  training.  The  LCRC  concept  cen^, 
around  the  judicious  use  of  embedded  PC-based  processors  an 
graphics  systems.  The  power  of  these  systems,  ^^ich 
commercially  available  is  phenomenal.  The  activity  described  .. 
based  on  a  study  involving  the  systems  engineering  evaluation 
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the  reauirenients  and  functional  aspects  of  the  reconf igurable 
"  V  sv^Tnwrrs  is  actively  pursuing  development  options  that 

e:S^?^lnitfaT;riead  ^a^dL^e  prototype  with  development  of 

the  software  in  Ada. 
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Interactive  Graphical  Models  in  Training: 
Authoring  Tool  Requirements 

Allen  Munro 
Douglas  M.  Towne 

University  of  Southern  California 


Computer  based  training  (CBT)  has  not  proven  itself  a  universal  panacea  for 
technical  training.  Worst-case  examples  of  CBT  provide  inflexible,  non-portable, 
low  resolution,  expensive,  demotivating  exp>eriences.  Best-case  examples  provide 
highly  motivating,  interactive,  simulation-based,  adaptive  instruction  —  but  such 
examples  are  ordinarily  very  expensive  to  develop  using  traditional  technologies. 

Our  laboratory  has  been  engaged  for  more  than  a  decade  in  the  development  of  a 
series  of  instructional  environments  with  integrated  authoring  tools  based  on  the 
principal  of  model-centered  computer  based  training  (CBT).  Model-centered  CBT 
systems  develop>ed  or  designed  at  our  laboratory  include  the  generalized 
maintenance  training  system  (GMTS),  the  equipment  simulation  authoring  system 
(ESAS),  the  intelligent  maintenance  training  system  (IMTS),  and  the  rapid 
intelligent  tutor  prototyping  development  system  (RAPIDS).  These  experiences, 
together  with  progress  in  human  interface  design,  have  led  to  the  formulation  of  a 
set  of  requirements  for  effective  and  economical  model-centered  CBT  authoring 
tools. 


ModeS-Centered  Computer-Based  Training 

In  model-centered  CBT,  students  interact  with  graphical  representations  of  complex 
devices  or  systems.  When  a  student  touches  a  pictured  switch  or  other  control,  the  pictured 
device  changes  to  reflect  the  effects  of  the  manipulation.  The  propagated  visual  effects  are 
determined  by  specifications  about  how  the  real  device  or  system  works.  These 
specifications  may  be  expressed  at  a  deep  level,  reflecting  the  actual  processes  involved  in 
the  device  behaviors,  or  at  superficial  levels  that  produce  the  visual  effects  without 
representing  the  internal  mechanisms. 

The  representation  of  a  device  can  consist  of  a  single  screen-sized  view,  called  a  scene,  or 
many  scenes.  The  representation  used  in  model-centered  CBT  could  be  an  interactive 
schematic  that  exhibits  internal  workings,  such  as  a  helicopter  bladefolding  system  (Figure 
1)  or  an  operational  front  panel  of  a  device  such  as  a  satellite  communications  system 
(Figure  2)  or  a  VCR  (Figure  3). 
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Figure  2.  One  Scene  From  a  Satellite 
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Figure  3.  A  VCR  Front  Panel  Training  Simulation 


Direct  Manipulation.  In  model-centered  CBT,  students  manipulate  elements  of  the 
graphical  model  in  a  manner  that  is  cognitively  congruent  to  the  way  that  a  real  device  is 
manipulated,  and  they  see  responses  of  the  device  that  are  either  realistic  or  instructionally 
meaningful.  When  the  student  clicks  the  mouse  pointer  on  the  right  contact  of  the  switch  in 
Figure  4A,  the  switch  changes  its  appearance  to  that  shown  in  Figure  4B.  The  ability  to 
change  the  appearance  or  location  of  an  object  by  clicking  on  it  or  dragging  it  is  called  direct 
manipulation. 

Automatic  Propagation.  When  the  switch  position  is  chang^,  other  effects  propagate 
through  the  graphical  model.  The  valve  changes  state  from  straight  to  crossed,  the  piston 
starts  to  move,  the  contact  switch  is  pushed  down,  and  the  light  turns  on.  This  sequence  or 
effects  results  from  the  direct  manipulation  of  the  switch  by  the  student  and  the  behaviorally 
defined  objects  that  are  connected  to  the  switch  direcUy  or  indirectly. 
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mese  tanas  oi  siuuciu  — o  -~o - .  -  _ r-oT 

evoking  them,  differ  markedly  from  the  simple  responses  pe^tted  m  co^yention^  CBT 
not  employing  an  active  model.  Thus,  m  our  experience,  model-centemd  Cm  otters  a 
significantly  enhanced  training  experience,  with  much  greater  potential  for  transfer  o 
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The  simplest  engine  °^h?Scraft  is 

oJTthe^grounl  TheTormal  way 

?his  engine  is  to  press  the  left  engme  start 

Kt*Se  Left  Start  Button  into  the  Pressed 
position. 

You  St'The  Left  Throttle,  which  was  the 
wrong  switch. 

You  set  the  Left  Timer,  which  was  the 
wrong  switch. 

The  Left  Start  Button  will  now  be  set  to 
Pressed. 


Click  anywhere  to  continue. 


f igarr  5.  a  f  araaa  »/./e  War.  Danas  «  Uodel.Cer,.r.d  TrmnU,t  E»rro. 


The  capabilities  rcguirrf  to  author  ^^fb^^^orfSidStreatioti  of 

representations  of  device  eleme^,  a  ^  for  developing  the  best  kinds  of 

instruction  based  on  the  model,  3  these  skills  to 

CBT  require  the  apphcanon  ft  r^iresKervices  of  professional 

create  interactive  sunulauon  training  is  labonou  ^  exnerts  in  training  specialues 

prograntmets.  Rarely  are  such  P'»8t«? 

or  in  the  particular  techntcd  domain  instructional-development  environment  can 

rsasSEK.rss'rsru.*.' 

In  the  remainder  of  this  pa^r,  we  PF““‘ “  course  planning, 

S^it?^Salo$S“SeTbehavior  authoring,  and  inshuctional  development 


Requirements  for  Course  Planning 

Cermin  taslts  must  ^  ^  TiS'S^mol^^vS  ^og^S^^r'  " 
developed  using  model-centered  authonng  .  hierarchical  structure  in  which 

methods.  In  RAPIDS,  authots  pta  n  ^  SS^fntwctional  presentations)  and 

performance  and/or  student  preferences. 
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RL  Aj. 

If  a  mcxiel-ceiitered  authoring  tool  is  to  have  utility  for  the  rapid  prototyping  of 
interactive  graphical  courses,  it  must  make  it  easy  to  build  a  course  that  simply 
consist  of  a  sequence  of  instructional  units.  Many  authors  who  are  not  experts  in 
theories  of  pedagogy  simply  want  to  have  their  students  carry  out  a  sequence  of 

exercises. 

R2.  A  non-programming  approach  that  provides  some  control  over  which 
instructional  units  are  presented 

The  next  step  up  in  sophistication  in  instructional  planning  is  one  that  makes  it 
possible  to  specify  that  a  module  of  instruction  should  not  be  presented  if  the 
student  meets  certain  criteria.  In  RAPIDS,  authors  can  go  through  a  sequence  of 


student  has  met  certain  performance  criteria  on  other  parts  of  the  course. 
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'a'ftorirgSd“ditmg*eooutseplan. 

environment.  .„  i^rruCioml  unis  from  a  d^eloping  course 

.  Authors  shouU  be  able  ror  er  -f  a  plan  tree  (terminal  nodes 

?XpIDS  authors  can  click  on  a  above)  in  “ 

p^art  of  the  course  plan. 

.  R6feanmer/ormu.iemenn"gJ|-'“^^S^^^ 

ta  aSition  to  having  “ op^^msde  i^ducn^JndSSd  be 
‘also  like  to  be  eM^'”Sn  S.  u  «ttm";S“afew  vtotato  occurs,  the 
is,  whenever  a  certain  ^  high-voltage  Z-gsented.  A  course 

m,—. »'  t;TZU.« 

SBSwS’.  *gs,“sss;..i.  »*“>  -»—**' 
SSSiSS-"*.'”—””- 

•-1,1,,  VmiIIi 


based  on  me  j  nreviously  tor  ouici  - - 

graphical  objects  produced  p  ^  ^  color, 

;S^ic°s'S?S-shS:n%elo«. 
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rei 

nply  as  a  bitmap  of  pixels,  bi 
objects  composed  of  elements  such  as  circles,  lines,  rectangles,  and  individual 
bitmaps.  Authors  must  be  able  to  copy,  paste,  and  delete  objects  and  collections  of 
objects,  as  well  as  change  the  order  in  which  objects  are  drawn  (moving  them 
from  back  to  front,  and  so  on),  and  change  their  intrinsic  graphical  attributes.  Such 
attributes  include  the  locations  of  the  objects,  the  line  thickness  of  those  objects 
that  are  made  up  of  line  components,  and  their  fill  colors  and  line  colors. 


This  requirement  is  a  shorthand  expression  of  a  large  number  of  human  interface 
design  rules.  Graphical  editors  should  behave  in  as  un-astonishing  a  fashion  as 
possible.  This  means  both  that  they  should,  as  far  as  possible,  reflect  the  behavior 
of  objects  in  the  real  world,  and  that  they  should  adhere  to  interface  standards  with 
which  users  may  already  be  familiar  (Apple  Computer,  1987;  IBM,  1989). 
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•  R9.  Graphical  power 

Two  types  of  graphical  power  are  required  for  successful  model-centered  CB 1 . 
First,  the  drawing  tools  provided  to  authors  must  provide  a  feature  set  that  makes 
possible  the  composition  of  expressive  and  effective  images.  This  means,  for 
example,  that  advanced  features  for  drawing  and  editing  curved  figures  may  be 
needed.  Similarly,  a  more  advanced  approach  to  color  choice  and  editing  than  a 
simple  sixteen-color  palette  will  be  necessary  for  many  training  applications. 
Second,  drawn  objects  must  be  rendered  quickly.  Many  CBT  models  will  deliver 
animation  effects  by  rapidly  erasing  and  redrawing  objects  in  model  scenes.  There 
is  a  natural  conflict  between  these  two  requirements  for  graphical  power.  Many 
CAD  systems  are  based  on  drawing  packages  that  permit  extremely  precise  and 
expressive  image  authoring  draw  much  too  slowly  for  CBT  use.  A  successful 
authoring  system  for  model-centered  CBT  will  balance  the  requirements  for  fine 
graphical  control  and  for  speed  to  produce  appropriate  performance. 

•  Importation  of  images 

Authors  will  often  be  able  to  prototype  a  CBT  trainer  or  even  build  a  complete 
model  using  images  that  have  been  scann^  from  existing  technical  or  training 
manuals  or  using  images  digitized  from  video.  Ideally,  a  model-centered  CBT 
authoring  system  will  not  only  support  the  importation  of  scanncdand  digitized 
images,  but  will  be  able  to  paste  graphics  that  have  been  created  in  other 
applications.  This  feature  will  make  it  possible  to  paste  graphics  that  have  been 
created  using  commercial  graphic  editors  that  support  standards  for  graphic  data 
export 


Requirements  for  Defining  the  Behavior  of  Component  Objects 

The  behavior  of  objects  must  be  defined  so  that  the  model  will  respond  appropriately  when 
model  elements  are  manipulated.  In  RAPIDS,  authors  can  write  rules  that  (tetermine  how 
objects  behave.  In  some  systems,  computer  programmers  write  complete  simulation 
programs,  and  then  link  aspects  of  the  appearances  of  objects  to  the  values  of  variables 
used  in  the  simulation  program. 


W  ((CurrentSi«t«  of  tolich  on  SIMPLEIA)  1«  'Clo»od) 
ttMn  (S«tS(«te  'On) 

•too  (SotScot*  'Off)  _ 


Figure  8.  A  Rule  that  Determines  the  Appearance  of  an  Object 

We  have  found  that  many  non-programming  individuals  can  learn  to  ^te  rules  that  control 
the  values  of  simulation  variables  called  attributes.  Some  of  these  attributes  determine  the 
appearances  of  objects.  If  the  simulation  variables  are  associated  with  particular  objects — 
that  is,  they  are  attributes  of  the  graphical  objects  —  then  authors  have  a  natural  way  of 
accessing  the  attributes  and  the  rules  that  govern  their  values. 
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®  R 1 1 .  Intrinsic  (especially  graphical)  attributes 

Many  changes  in  the  appearances  of  graphical  objects  can  be  expressed  in  terms  of 
the  attributes  of  some  of  the  graphical  elements  of  the  model.  For  example,  the 
value  portrayed  by  a  meter  may  be  expressed  in  terms  of  the  rotation  of  the  meter’s 
needle.  The  height  of  a  bar  on  a  bar  chart  can  be  expressed  in  terms  of  the  length 
of  the  bar.  Attributes  like  length  and  rotation  are  intrinsic  attributes.  Evesy 
graphical  element  has  the  location  attribute.  Line  type  elements  have  length,  line 
thickness,  and  line  color.  Authors  can  control  the  appearances  of  graphical 
elements  by  writing  rules  that  determine  the  values  of  these  intrinsic  graphical 
attributes.  Whenever  one  of  these  attributes  changes  in  value,  the  object  is 
redrawn. 

®  R12.  Author-d^ned  attributes 

Authors  must  not  be  restricted  to  using  intrinsic  attributes  in  their  simulations. 
When  an  economist  creates  a  model  for  training,  he  or  she  may  create  a  ‘gross 
domestic  product’  attribute.  When  a  hydraulics  maintenance  instructor  develops  a 
CBT  application,  he  or  she  may  define  a  new  attribute  called  ‘Port  C2  input 
pressure’  for  a  particular  valve. 


®  R 13.  Behavior  authoring  without  programming  the  flow  of  control 

While  many  non-programmers  can  write  rules  that  determine  the  values  of 
attributes,  it  is  more  difficult  for  most  of  them  to  create  program  control  structures 
that  determine  the  order  in  which  such  rules  are  evaluated.  One  of  the  tasks  that  a 
model-centered  CBT  simulation  must  carry  out  for  the  author  is  to  determine 
behind  the  scenes  how  simulation  effects  should  propagate  among  attributes  that 
can  affect  each  other. 


®  R14.  A  natural  expression  syntax 

The  expression  of  value-determining  rules  should  be  as  natural  and  as  re^ble  as 
possible.  Iconic  representations  of  logical  and/or  mathematical  relationships  seem 
to  be  less  clear  for  most  authors  than  using  words  like  ‘if  and  ‘then.’ 

®  R15.  Rules  for  responding  to  student  actions 

The  rule  syntax  must  have  natural  ways  of  referring  to  student  actions,  such  as 
clicking  on  an  object 

®  R16.  Modeling  real-time  processes 

In  addition  to  providing  an  expression  syntax  for  assigning  values  to  object 
attributes,  a  model-centered  system  for  technical  training  should  facilitate  the 
authoring  of  time-dependent  processes  that  change  attribute  values  gradually,  once 
they  have  been  activated.  It  should  also  be  possiWe  to  author  rules  that  interrupt 
these  processes. 

®  R17.  Scheduling  events 

Rules  should  be  able  to  schedule  events.  This  makes  it  possible  to  write  a 
simulation  that  explodes  a  bomb  at  a  fixed  time  after  its  fuse  is  lit.  Rules  must  also 
be  able  to  deschedule  events.  This  makes  it  possible  to  simulate  the  fuse  being 
extinguished. 

®  KIZ.  Aid  in  rule  authoring 

A  rule  authoring  system  should  offer  many  types  of  support  for  model  authors. 

Menu-based  rule  authoring.  It  should  be  possible  for  an  author  to  select  a  rule  or  a 
portion  of  a  rule  and  ask  for  a  menu  of  options  for  editing  that  rule.  The  menu 


Interactive  Graphical  Models  in  Training:  Authoring  Tool  Requirements 


January  14-16. 1992 


should  be  context-sensitive,  so  that  it  offers  only  syntactically  correct  alternatives 
for  the  selected  rule  segment 

Authoring  certain  types  of  rules  graphically.  In  RAPIDS  autto  ^"^le  to  create 
simple  rules  that  assign  the  value  of  one  attribute  to  another  attnbute  sunply  by 
making  a  series  of  menu  choices  and  mouse  clicks  on  the  two  objects. 

•  R19.  Interactive  model  debugging  features  thHr 

Most  complex  models  are  likely  to 

development  Authors  sometimes  wnte  rules  that  don  t  carry  out  the 
S  JSts  oXy  fail  to  write  planned  rules,  so  that  no  vdue  is  establish^  for 

an  aSibute.  A  number  of  debugging  features  c^  sign^ificantly 

of  the  model-au  Jioring  environment  These  include  the  foUowmg  optio 

features;  .  ^ 

-  Dialog  views  of  object  and  attnbute  data 

-  Graphs  of  attribute  effect  propagation 

-  Pausing  on  rule  evaluation 

-  Pausing  on  attribute  value  changes 

-  Stepping  through  rule  evaluations  _ _ 

-  cScting  attribute  value  ennrs  (e.g.,  ‘no  value  provided  )  amin  tt 

Requirements  for  Authoring  Instruction 

In  RAPIDS  instructional  units  are  authored  primarily  by  directly  (^crating  the 
mcSS^Sr  e^^rrauto  can  create  a  r^uired  student  identification 
a  ‘Find  Object’  option  from  a  menu  of  instructional  item  ihen  ctong  on 

object  that  Ae  student  is  to  find.  A  student  prompt  (e.g..  Find  the  Left  f  “^ton  ) 
presented  automaticaUy  as  a  result  of  these  simple  S 

Siteractions  with  the  smdent  including  delation  and 

similar  manner,  procedures  are  demonstrated  and  explamed  by  performing  them  on  me 
model. 

•  R2Q  Creating  reauired  student  actions  by  performing  the  (prions  _ 

If  a  student  is  to  be  required  to  click  on  an  object  as  part  of  the  exercise,  auAormg 
^t  rSTiS^m^t  sho^d  include  cUcking  on  the  object  If  Ae  stitde."t 
the  sS  of  a  switch,  then  the  author  should  be  able  to  auAor 
bv  clicking  on  the  new  switch  position.  As  far  as  is  possible,  prorripts  to 
presOTted  to  the  student  should  be  automatically  composed  rather  than  author 
?SnTOsed.  Naturally,  additional  features  are  required  for  letting  ^e  author  ^^^ect 
SSexplanaJ^y  materials,  but  much  of  the  drudge  work  of  bmldmg  simple 
simulation  excises  can  be  automated  m  a  model-centered  authonng  system. 

•  R21  Recording  and  restoring  system  states 

To  be  contextiully  appropriate,  many  instructional  sequences  will  r^aire  that  tiie 
model  be  in  a  particiSSr  state  when  that  instructional  umt  is  present^.  It  ^ould  be 
a  straightforwLi  matter  for  authors  to  record  such  system  states  and  to  specf  y 
which^one  is  to  be  used  with  each  instructional  unit.  At  presentation  time,  the 
recorded  system  states  must  then  be  loaded  automaPcally. 

•  indudjng  id^ndficadon 

SSIW  exercisesVdaining  to  f  “S’, 

operation  exercises,  indicator  reading  exercises,  procedure  exercises,  and  a  vanety 
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of  fault  diagnosis  exercises.  A  powerful  model-based  authoring  system  should 
provide  the  tools  for  composing  such  exercises  by  simply  indicating  which  parts 
of  a  device  or  procedure  are  to  be  covered.  Such  authoring  tools  should  include 
prompts  to  the  author,  thereby  minimizing  the  requirements  for  learning  to  author 
instruction. 

®  R23.  Free-play  model-centered  exploration 

Once  a  working  model  has  been  developed,  an  author  can  make  that  model 
available  to  the  student  in  a  free-play  mode,  so  that  students  can  explore  their 
understanding  of  the  operation  of  the  portrayed  system. 

®  R24.  Building  instructional  templates 

If  a  new  kind  of  exercise  is  developed  by  the  author  that  involves  the  repetition  of 
a  particular  sequence  of  instructional  item  types,  the  author  should  be  able  to  create 
an  instructional  macro  or  template  that  will  automate  the  process  of  developing 
exercises  of  this  new  kind. 


As  mc^l-centered  authoring  systems  are  used  to  develop  operator  training,  maintenance 
training,  and  system  familiarization  training  courses,  a  number  of  additional  requirements 
are  likely  to  emerge.  Some  of  these  may  include  more  built-in  intelligence  about  the  task 
requirements.  For  example,  a  model-centered  system  for  authoring  maintenance  training 
could  have  a  built-in  expert  that  could  evaluate  the  utility  of  diagnostic  actions  (Towne, 
Munro,  Pizzini,  Surmon,  Coller,  and  Wogulis,  1990). 

Future  systems  may  be  able  to  support  more  elaborate  approaches  to  imaging  and  user 
interaction.  In  3D  graphics  models,  for  example,  there  would  be  additional  graphical 
primitive  elements,  such  as  rectangular  prisms,  cones,  and  spheres.  These  new  types  of 
elements  would  have  additional  intrinsic  attributes  that  would  permit  control  of  their  depth, 
their  pitch  mid  yaw,  and  their  location  in  the  Y  plane  of  space. 

More  dramatically,  virtual  reality  systems  not  only  offer  potential  for  3D  graphical  attribute 
control,  but  also  for  enhanced  user  action.  Using  a  data  glove,  students  will  not  be 
restricted  to  clicking  and  dragging  operations,  as  they  are  with  a  mouse,  but  will  be  able  to 
grasp,  push,  and  pull.  Although  these  kinds  of  changes  will  call  for  enhancements  in  the 
graphical  composition  tools  and  in  aspects  of  the  behavior  rule  syntax,  they  should  have 
less  impact  on  the  requirements  for  course  planning  and  for  instruction  development 
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Figure  2.  One  Scene  From  a  Satellite  Communications  System  Model 


324 


Interactive  Graphical  Models  in  Training:  Authoring  Tool  Requirements 


January  14-16, 1992 


325 


®5!j=!  Ob 


Interactive  Graphical  Models  in  Training:  Authoring  Tool  Requirements 


January  14-16, 1992 


Interactive  Graphical  Models  in  Training:  Authoring  Tool  Requirements  January  14-16, 1992 


Exit 

Save 

AddUnIt 

DeleteUnIt 

l^oveToFarent 

IdoveToTop 

tdoveToNode 

SetDepth 

RepositlonUnit 

'  \  '^*dv4nc>»4  ♦Dtort  dh#«  Dddd  In  Air  Intra 

\  L«ft  Drill; 

\Dd.r,t1.n  Drink:^-"™-'^'™^^ 

CouraeFarams 

''"^Advonco^^rinl  iDtort  ihon  Oood  In  Air  Drill: 

Name:  ClMant<ry  Drill 

Commeot; 

Order  at  preientatloa:  xtndai  ■iiumjiaita 
Content: 

Unit _ Weight  Mode  Condition  Maximum  Minimum  Limit  Accuracy  Speed 

St«rt  L«ft  Orl  1  Drill  Not 

St4rt  Klfht  Or  1  Drill  Hoc  dofIrMd 


Figure  6.  A  Course  Plan  for  RAPIDS 


fin 

W 


327 


Micro-Video  Display  with  Ocular  Tracking 
and  Interactive  Voice  Control 

(Sensory  Integrated  Data  Interface  (SIDI)) 


James  E.  Miller 

Naval  Underwater  Systems  Center 
Newport,  RI  02841 

December  9,  1991 


Abstract:  An  in-process  effort  to  develop  a  highly  miniat¬ 
urized  solid  state  computer  display  and  control 
interface  system  is  described.  The  system  incor¬ 
porates  a  micro-video  display  with  ocular  tracking 
on  a  lightweight  headset  and  is  controlled  by 
voice  commands  from  the  operator.  Various  "off- 
the-shelf  components  are  integrated  into  a  PC 
compatible  computer  along  with  a  prototype  hyper¬ 
text  software  application  to  demonstrate  conceptu¬ 
al  capabilities.  This  effort  is  Phase  I  of  Small 
Business  Innovative  Research  (SBIR)  Topic  #N90-331 
sponsored  by  the  Naval  Air  Systems  Command.  The 
prime  contractor  is  Foster-Miller,  Inc. 


Introduction 

The  last  decade  has  produced  an  explosive  growth  of 
computer  miniaturization  technologies  which  continues  un¬ 
abated  to  this  day.  But  while  these  technologies  have 
reduced  computer  size  and  improved  processor  speed,  memory, 
and  software  capability  by  several  orders  of  magnitude,  user 
interface  improvements  have  been  relatively  few.  Virtually 
all  computer  systems  still  rely  on  large  and  cumbersome 
interface  devices  -  namely  CRT/LCD  screen  displays  and  key¬ 
boards  -  with  relatively  little  technological  improvement 
over  their  1950s  predecessors. 
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able  which  can  provide  significant  improvement  for  the 
himan-to-computer  interface.  Each  one  of  these  technologies 
singularly  can  provide  improvement,  but  significant  synergy 
may  be  gained  by  combining  several  of  them  together  into  a 
single,  integrated  system.  The  goal  of  this  project  is  to 
prototype  four  of  these  technologies  into  an  effective 


interface  system  for  the  personal  computer. 
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This  effort  will  attempt  to  assemble  an  interactive 
interface  consisting  of  a  micro-video  display  in  which 
screen  cursor  movement  follows  eye  movement  and  menu  func¬ 
tion  selection  is  implemented  by  voice  command.  The  display 
projector/ eye-tracker  will  be  worn  on  a  lightweight  headset 
in  a  monocular  arrangement  with  a  small  microphone  incor¬ 
porated  for  voice  interaction.  An  existing  hypertext  soft¬ 
ware  application,  Computer  Aided  Fault  Isolation  (CAFI)  for 
the  Encapsulated  HARPOON  Certification  and  Training  Vehicle 
(EHCTV)  will  be  integrated  with  the  hardware  to  demonstrate 
conceptual  capabilities. 

The  newly  developed  technologies  to  be  applied  in  this 
effort  are  those  of  micro-video  displays,  ocular  tracking 
systems,  computer  voice  recognition,  and  hypertext  software 
applications.  Based  upon  a  survey  of  available  products  on 
the  market,  the  systems  described  below  have  been  selected 
for  use  in  this  project.  High  level  diagrams  of  the  inte¬ 
grated  system  are  shown  in  Figures  1  and  2 . 

Micro-Video  Display  System 

The  system  chosen  to  display  computer  data  is  the 
"Private  Eye."  The  system  consists  of  a  small,  matchbox¬ 
sized  projector  unit  mounted  on  a  lightweight  headset.  The 
projector  is  worn  in  a  monocular  position  in  front  of  one 
eye  and  interfaces  to  a  standard  PC  computer  data  bus  via  an 
interface  card.  A  virtual  image  of  the  computer  CRT  display 
is  projected  approximately  18  inches  in  front  of  the 
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The  hardware  functions  by  using  a  custom^  intS"” 
grated  circuit  chip  set  and  tracking  algorithms  to  follow  a 
point  on  the  eye  in  real-time <>  It  accomplishes  this  by 
measuring  and  tracking  a  low-level  infrared  beam  reflected 
from  the  eye  surface c  A  miniature  infra-red  video  camera  is 
used  to  acquire  the  pupil  and  corneal  images  needed  by  the 
tracking  algorithms =  This  tracking  function  allows  movement 
of  a  screen  cursor  based  on  eye  movement/posit  ion,,  that  is, 
the  cursor  moves  to  where  the  operator  ”  looks  »'*  Both  the 
ESP  and  Private  Eye  are  mounted  on  a  lightweight  and  adjust¬ 
able  headset  which  can  accommodate  variable  positioning  of 
the  components. 


Computer  Voice  Recognition 

The  voice  recognition  unit  chosen  to  provide  voice 
interaction  with  the  operator  is  the  Convex  Voice  Master. 

The  system  consists  of  a  control  unit,  speaker,  microphone, 
and  software  which  can  provide  both  speech  recognition  and 
generation  capability  for  a  PC  compatible  computer  using  a 
RS”232  interface.  The  operator  will  use  voice  commands  to 
implement  menu  options  once  the  ESP  has  placed  the  display 
cursor  on  them  for  selection. 

Hypertext  Software  Applications 

An  existing  hypertext  application  developed  by  Naval 
Underwater  Systems  Center  (NUSC)  was  chosen  for  implementa¬ 
tion  in  this  project.  This  application.  Computer  Aided 
Fault  Isolation  (CAFI)  for  the  Encapsulated  HARPOON  Certifi¬ 
cation  and  Training  Vehicle  (EHCTV)  was  developed  in  Guide 
2.0  (a  commercially  available  hypertext  development  environ¬ 
ment)  .  The  program  allows  the  operator  to  access  trouble¬ 
shooting  data,  schematics,  failure  databases,  and  repair 
procedures  for  the  EHCTV  by  simply  pointing  and  clicking  on 
screen  "buttons . " 

Project  Status 

Project  work  is  in  progress  having  been  started  in 
October  1991.  The  system  base-line  design  has  been  estab¬ 
lished  and  hardware  procurement  is  complete.  Work  current¬ 
ly  underway  includes  development  of  special  system  software 
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Tactical  Naval  Applications  of  Advanced  3-D  Display  Technologies 


David  Rousseau 

Naval  Ocean  Systems  Center,  San  Diego,  CA 

Introduction 

A  group  of  rapidly  maturing  technologies  are  coming  together  to  permit  a  new 
way  of  displaying  and  controlling  information.  The  increasing  speed  and  decreasing 
cost  of  high-resolution  graphics  computers  is  making  it  practical  to  generate  complex 
imagery  with  acceptable  realism  in  real  time.  Small  high-resolution  monochrome 
cathode  ray  tubes  (CRTs)  have  long  been  used  for  military  helmet-mounted  displays. 
Liquid  crystal  displays  (LCDs)  with  acceptable  resolution  are  just  now  becoming 
available.  Both  are  finding  their  way  into  commercial  stereoscopic  display  systems. 
Commercial  versions  of  position  sensors  developed  for  military  head-trackers  are 
becoming  increasingly  affordable,  and  their  use  has  been  expanded  to  create  three- 
dimensional  (3-D)  controllers.  Finally,  the  newest  technology  on  the  scene  is  digitally 
modified  sound  that  presents  the  operator  with  realistic  3-D  binaural  hearing.  These 
3-D  and  other  display  technologies  are  being  studied  for  their  application  to  tactical 
Naval  operations  at  the  Naval  Ocean  Systems  Center  (NOSC).  Among  the  potential 
beneficiaries  of  this  technology  are  airborne  early  warning  and  forward  air  control, 
platform  and  force  level  battle  management,  air  traffic  control,  compact  flight  trainers 
and  mission  planners,  and  many  aspects  of  anti-submarine  warfare  (ASW).  The 
mission  area  currently  being  investigated  at  NOSC  is  in  the  improvement  of  shipboard 
ASW  sensor  information  displays. 


Discussion 

It  is  known  that  performance  with  displays  improves  when  operator  workload  is 
reduced^'  2-  3  and  that  this  effect  is  amplified  by  prolonged  operations.^  This  is 
because  reduction  in  operator  workload  frees  up  cognitive  resources  which  can  then 
be  applied  to  other  tasks  such  as  improved  decision  making.Ls  Therefore,  displays 
which  minimize  this  work  load  should  result  in  improved  operator  performance  and 
improved  decision  making.  The  need,  therefore,  is  for  displays  which  present 
information  in  ways  which  take  advantage  of  natural  human  perceptual  and  cognitive 
skills.  Few  display  systems  in  use  today  are  more  abstract  than  those  used  to  support 
the  ASW  mission  area. 

Sonar  displays  have  not  changed  significantly  for  roughly  ten  years.  Existing 
ASW  displays  present  information  that  is  frequently  truncated,  altered,  or  otherwise 
simplified  due  to  limitations  in  data  processing,  or  the  physical  limitations  of  the 
display  devices  themselves.  Currently,  diverse  sources  of  information  must  be 
correlated  by  the  sonar  operator  who  creates  a  mental  model  of  the  multi-dimensional 
ASW  environment.  The  majority  of  this  mental  image  must  be  derived  from  multi¬ 
dimensional  acoustic,  physical,  and  temporal  information  that  has  been  presented  in  a 
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very  abstract  visual  two-dimensional  {2-D)  format  from  a  number  of  sources  (Figure 
1).  This  method  of  presenting  complex,  cross-sensory,  multi-dimensional  information 
on  abstract  2-D  displays  may  soon  become  obsolete.  The  application  of  new  3-D 
visual  and  audio  display  technologies  to  this  problem  could  yield  significant 
improvements  in  operator  performance  and  operational  effectiveness.  These  3-D 
systems  could  present  the  critical  tactical  information  in  a  far  more  intuitive  and 
integrated  way,  thereby  reducing  the  cognitive  interpretation  burden  and  learning  time, 
while  improving  the  tactical  success  rate  over  current  systems.  Figure  2  depicts  the 
nature  of  the  relationship  between  cognition,  response  time,  and  error  rate.  When  the 
task  of  interpreting  information  is  complicated,  the  response  or  reaction  of  the 
individual  will  be  slower  and  the  error  rate  will  increase. 

An  advanced  technology  ASW  display  system  could  consist  of,  1)  the 
application  of  3-D,  stereoscopic,  high-resolution,  helmet-mounted  or  boom-mounted 
displays,  2)  a  3-D  position  tracker  (for  the  HMD)  and  3-D  manipulators  for  computer 
function  control,  3)  the  incorporation  of  3-D  audio  for  presentation  of  multiple  beams 
from  the  sonar  for  intuitive  cuing  and  for  correlation  of  acoustic  transients  and  active 
returns  to  the  ocean  environment,  and  4)  the  intuitive  depiction  of  high-resolution 
computer-generated  imagery  of  the  ASW  environment  based  on  integrated  display  of 
active,  passive,  and  environmental  sensor  Information.  The  exact  nature  of  the 
displays  that  may  prove  most  effective  are  yet  to  be  determined,  but  speculation 
unfettered  by  display  constraints  leads  to  some  intriguing  possibilities. 

For  example,  in  the  active  sonar  mode  the  3-D  ASW  display  could  provide  the 
operator  with  a  360°  field-of-regard  image  of  the  ocean  bottom  topography  and  water 
properties  integrated  with  wave  fronts  representing  the  propagation  of  the  sonar 
pulses  (Figure  3).  The  operator  would  see  these  wave  fronts  being  reflected  off  of  the 
local  topography  and  all  known  obstacles  in  that  data  base,  and  refracted  by  the 
mode!  of  the  local  water  properties.  The  operator  would  therefore  be  able  to  see 
which  returns  are  from  known  obstructions  and  compare  that  with  the  actual  returns 
received  by  the  sonar.  Integrating  this  3-D  visual  representation  with  acoustic  analysis 
and  cuing  from  the  3-D  audio  system  would  give  the  operator  an  intuitive  picture  of 
the  ASW  situation.  Similarly,  in  the  passive  sonar  mode  the  operator  would  be  able  to 
look  around  at  real-time  noise  sources  with  cuing  from  the  3-D  audio  system  (Figure 
4).  He  could  also  view  the  sonar  history  over  a  desired  time  span  for  all  bearings  and 
elevations  in  a  single  3-D  volume  (Figure  5).  In  each  case  any  sonar  beam  could  then 
be  designated  for  spectra!  analysis. 

Similar  display  approaches  could  be  used  for  fixed  site  ASW  facilities. 
Application  of  such  a  display  system  to  airborne  ASW  is  more  challenging  due  to  the 
strict  size  and  weight  limits  of  ASW  aircraft,  but  computer  size  and  weight  are 
constantly  decreasing.  Many  of  the  displays  developed  for  surface  ASW  would  be 
directly  applicable  to  both  airborne  and  ashore  ASW.  A  few  specialized  displays  may 
be  added  for  such  things  as  sonobuoy  coverage  however  (Figure  6). 


Significant  enhancements  to  total  system  effectiveness  could  result,  without 
redesign  of  other  system  components  (e.g.,  sensors  and  sources  of  self  noise). 

Recent  experiments  measured  significant  changes  in  sonar  operator  performance  due 
to  relatively  minor  modification  of  a  standard  sonar  display.^  These  experiments  were 
conducted  under  ideal  signal-to-noise  conditions  and  measured  a  multiple  db  loss 
due  to  a  proposed  fusion  of  data  on  a  conventional  ASW  display. 

As  this  research  continues,  new  methods  for  presenting  ASW  information  could 
arise  that  would  make  data  relationships  more  apparent,  increase  situational 
awareness  and  enhance  the  quality  of  operator  problem  solving.  This,  in  turn,  may 
enable  development  of  more  sophisticated  and  effective  tactics.  Certainly,  current 
tasks  of  target  detection,  localization,  and  classification  would  improve.  Reductions  in 
operator  training  and  increases  in  retention  could  also  be  significant,  and  the 
effectiveness  of  each  sonar  operator  could  be  multiplied  as  well. 

This  technology  could  also  be  applied  effectively  to  ASW  tactical  support  by 
providing  the  shipboard  ASW  officer  (or  submarine  approach  officer.  Figure  7)  with  a 
3-D  image  of  the  ocean  environment,  his  weapons  engagement  envelops,  sensor 
coverage  volumes,  and  the  hostile  submarines  that  have  been  identified. 

3-D  display  technology  could  also  provide  E-2C  controllers  with  3-D  aircraft 
engagement  information  including  launch  envelopes,  detection  system  coverage,  etc. 
(Figure  8). 

It  could  be  applied  to  the  display  of  battle  management  information  (Figure  9) 
where  location  and  status  of  assets  determines  weapons  and  sensor  coverage. 

A  simpler  application  would  be  for  air  traffic  control  afloat  or  ashore  (Figure  1 0) 
where  only  limited  organic  sensor  information  must  be  presented.  Such  a  system 
might  also  benefit  from  the  incorporation  of  real-time  voice  control  of  the 
communications  system. 

The  Technologies 

Four  technologies  are  maturing  sufficiently  to  permit  a  serious  investigation  of 
their  operational  use  in  an  integrated  information  display  system.  They  are  high¬ 
speed  graphics  computers,  miniature  high-resolution  displays,  3-D  position  sensors, 
and  digital  3-D  audio. 

COMPUTERS 

High-speed  graphics  computers  are  capable  of  generating  acceptably  realistic 
imagery  that  is  updated  fast  enough  to  present  an  operator  with  useful  information  in 
real  time.  In  the  past,  such  capabilities  were  limited  to  massive  and  costly 
supercomputers.  Fortunately  the  highly  competitive  nature  of  this  industry  continues  to 
push  cost  down  and  performance  up.  Present  high-speed  graphics  computers 
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surpass  the  performance  of  these  older  machines  at  a  fraction  of  the  cost  and  size.  A 
few  years  ago  a  small  computer  that  could  run  at  5  million  floating  point  operations  per 
second  (MFLOP)  and  generate  about  6,000  polygons  per  second  was  the  hottest  thing 
on  the  market.  Today  that  would  be  a  “low-end”  machine.  The  “high-end”  graphics 
computers  today  run  about  30  times  faster  (180,000  polygons  per  second)  at  a 
comparable  price.  All  of  these  machines  are  using  multiple  processors  to  achieve 
these  speed  and  cost  improvements  with  large  supercomputers  now  operating  with 
thousands  of  processors.  The  next  ten  years  will  see  a  continued  increase  in 
performance  with  a  corresponding  decrease  in  cost  and  size.  Today’s  high-speed 
graphics  computers  will  seem  slow  by  the  time  a  3-D  display  system  is  ready  to  enter 
the  Fleet. 

DISPLAYS 

Miniature  high-resolution  displays  evolved  from  weapons  control  and  sensor 
display  requirements  for  military  aircraft.  Much  of  this  work  was  pioneered  at  the  Naval 
Weapons  Center  at  China  Lake  under  the  AGILE  Missile  program  and  by  the  Air  Force 
at  Wright-Patterson  AFB.  Although  the  Navy  fielded  the  first  operational  helmet- 
mounted  display  (HMD),  the  most  widely  used  of  these  systems  can  be  found  aboard 
combat  helicopters.  These  systems  now  use  cathode  ray  tubes  (CRTs)  as  small  as  0.5 
inches  in  diameter  with  1,000  line  resolution.  Approximately  875  to  1000  line 
resolution  is  required  for  “seamless”  imagery  in  an  HMD  with  an  average  field  of  view. 
By  way  of  comparison,  a  photograph  is  approximately  4,000  pixels  high.  All  of  these 
small  CRTs  are  either  monochrome  or  black  and  white.  There  appears  to  be  a 
technology  barrier  against  making  full-color  CRTs  of  this  size,  although  there  has  not 
been  a  strong  military  or  commercial  requirement  to  push  small  color-CRT  technology. 

Small  monochrome  CRTs  from  portable  TVs  are  in  use  in  boom-mounted 
stereoscopic  displays.  Unfortunately,  the  process  of  magnifying  the  imagery  from 
these  CRTs  for  a  stereoscopic  display  with  an  average  field  of  view  causes  resolution 
problems.  The  use  of  standard  TV  technology  (550  line  NTSC  video)  results  in  very 
noticeable  black  stripes  across  the  image  due  to  the  return  stroke  blanking  used  in 
conventional  raster  scan  video.  The  use  of  high  definition  TV  would  reduce  or 
eliminate  this  problem,  but  the  day  that  such  displays  are  available  in  the  small- 
portable  market  is  a  long  way  off. 

Color  active-matrix  Liquid  Crystal  Displays  (LCDs)  for  the  miniature  TV  market 
are  now  being  used  for  displays  in  the  majority  of  the  commercial  stereoscopic 
systems.  These  displays  have  recently  doubled  in  linear  resolution  to  the  point  of 
being  comparable  with  standard  television  resolution  (LCD  image  height  of  480  pixels 
vs.  500  lines  of  raster-scan  video  for  conventional  TV).  Although  this  is  a  great 
improvement  over  the  previous  generation  of  240  pixel-high  LCDs,  this  resolution  is 
still  low  enough  that  the  image  appears  very  grainy  when  magnified  as  part  of  a 
stereoscopic  display  system.  Unfortunately,  higher  resolution  LCDs  are  not  likely  to 
emerge  from  the  commercial  market  unless  the  demand  increases  by  several  orders  of 
magnitude.  The  technology  for  creating  a  1.0  inch  diagonal  1 ,024  x  1 ,024  pixel  active- 
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and  a  field  sensor  attached  to  the  ^^6  to  of  fhe  sensor  based 

monitors  the  field  sensor  P  =  ^ujoh  it  is  immersed.  These  units 

on  the  nature  of  the  portion  of  the  .  oresence  of  ferrous  metals  which 

are  small  and  rugged,  but  they  are  seosit  P  reduce  this  problem 

can  distort  the  magnetic  system  Other  p^osition  trackers  are  now  reaching 

and  to  speed  up  the  operation  of  the  syste  .  P  ultrasonic  sensors  and 

^s^ourceTTo^r^^^^^^^^ 

^  L"f,heir  ™“!iSTs."The  Polhemus  system  is  the  dniy  one  ,ha,  is 
currently  incorporated  into  3-D  control  devices. 

3-d  audio 

human  auditory  perception.  •  •  counds  Subsequently,  the  ability  to  modify 

shape  of  the  outer  ear  to  the  The  transform 

sounds  using  expenmentally  ..  .  rtion  of  known  broad-band  sounds  due 

functions  were  derived  by  measunng  function 

to  outer  ear  shape.  An  undistorted  desired  location. 

I  incorporated  into  circuity,  would  seem 

"  This  made  it  possible  to  alter  ^  sound  so  f{ecenX  work  at  NASA,  Ames  resulted 

chosen  location  in  space  into  dioital  circuity  and  married  to  computers 

Sor  the  on®nt^ 

conducting  research  into  its  application  to  various  operational  tasks. 

Conclusions 

These  .echnoipgies  are 

including  architecture,  "'®‘^PP™Pf' P  „ 3.0  display  dt  CAT  scan,  NMR,  and 
,  SrcrrdlcaTdX^^^^^^^^^  “r.  aldslr^the  handicapped. 
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If  these  new  technologies  are  applied  to  the  tactical  Naval  environment  their 
impact  on  operational  effectiveness  could  be  dramatic  and  their  impact  on  training 
methods  and  systems  may  be  equally  dramatic. 

History  has  taught  us  that  the  requirement  to  °P®'f  ®  ® 
diminish  at  a  rate  commensurate  with  the  decrease  in  our  assets.  Budgets  and 
Lnpower  levels  will  probably  continue  to  decrease  °^®'  'j!®  °  *  s^g 

Therefore  the  workload  for  each  person  will  increase,  and  the  tactical  co  9 

one  platform  out  of  a  reduced  inventory  will  be  magnified. 

operational  pertormance  while  requiring  lower  manning  P®  ’3“®  3, 

systems  that  can  achieve  the  greatest  performance  improvements  at  the  lowest  cost 

will  be  essential. 


1.  Trejo,  L  J.,  Lewis,  G.  W., 


H.  (1990).  Brain  Activity  During  Tactical 
“  ■  Simulation 


Multidimensional  audio  information  is 
presented  as  abstract  visual  data  in  2-D 
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INTRODDCTIOS  -etrievci.  r.c  ce/.very 

The  Artificial  Intelligence  -  Explosive  0^^ Srci^rLiLpnent ^  'S:vf Licalce 

Ilifcrnia:  the  'llnrylnnc.  For  further  information  see  .on.e.,  . . 

Disposal  Technology  Center  iNECDii), 

Hollsnd  (1992) ■  .  ,  «Qn  -- 

rrir 

tecUici...  Increitei.  .  jiliitiltj  o. 


formance  in  the  held.  E  ,/eilor,prone  and  labor  intensive  chore. 

'TKl'pcee/goals  of  the  fJ;tTovide^^S^^^ 

search  information  and  retrieval  sys  em  °nab  ity  to  improve  the  identification  spee  and  accuracy, 

Specification  (Development  Option  Paper). 

m-bod  system  FODHDATIOH 

2sflf=’SSs^ss=l 

«■«»•  '>«»•  ‘‘““'““S’;"' 

cations  (Conner  et  al.,  1592). 

I  , .  ™ 

The  Al-KOD  prototype  is  an  AI  baseo  mu  v 
state-of-the-art  image  and  text  management. 
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The  prototype  system  was  Sun  4  5FARC  workstat:::..  ^The 

and  use  of  available  software  tools,  .he  ..  ■■  interface  (GDI1  uses  the  Open  -oc<  preseruc..-  - 

:;su.  is  mn  os;,  s:s.is:i  ;0;;; 

Sir;;;S;o»;s-;»  loss.  M  .nion  .i  ..  o.^.,  ,0.  o..rs;  osi,  «s  . . 

data  are  expected  to  be  straight  ASC . as. 

Software 

The  software  is  separated  into  physical  resources  of  the  cor, outer.  ^ 

1.  The  operating  system,  which  manages  t  ®  r  the  computer  to  achieve  the  users  goals. 

„’jro'“u:r:‘:u.  r'/”;hfo..:T.rL c.  o.  .oi..-  o,  to.  osoo, 

,;rj:fs;:,’i[S.”:;ET;os-b.L  ...o,..  o,  m....  .,s...  o..  .0. ...... ».  .... 

i.  ,J  to.  .s.r  'thinks'  is  th.  srst...  It  “  n,  /..joD  srst..  ..st  h.,..  1 

ed-  lt-«indoHS  piosides  a  haana  to  present  a  t  .  Phe  EOD  technicians  oust  hane  the  abi.  ., 

ced  to  piooeed  with  a  rigid  aegnence  oi  qnestions  and  ansaeis.  in.  the  ptoc  - 

Snd  to  <hMl.e.  sitnatio.  -1«  *"1  ,•  that  Is  d«  no?  o"**""'* 

-'har:r.si siihirariSsra^pSL . r  thMr..:;"a‘or't;:  l.; 
'T'Se trt:.crs.«t“ru« 

coin  group  4  data  cospression  is  recoaaended.  interest. 

5'  lir,’ra‘r'aa’'.rl..et..t  It  he  e!.he.  h,  dte.i.g  a  hoa  aroaad  th.  at.,  ol  lataias. 

*  "f^S'raef  .Sdr.'hf.‘r.-..dlt,  th.  « 

SS  usSi'doS  ..'til’ntraight  U.aa  are  st.l.a.appad  tro.  Pi.al  hi.«  aP- 

th.  i-f- -“.“-itaSr'a?  tan  m-a  ««  -‘'r^ 

;!  ha’:r.'ed”'.uh  rupiit,r.;;*'  ..ch  ida.ti.,i.g  th.  p.hiic.tioh ... ..... 

-  1  _L  n  r\  JoM^AAP 


5. 

6. 
7. 


number. 


10. 

11. 

12. 

13. 


Bsats  ..at  h.  .hi.  to  rotate  th.  inege  i.  th.  Lag.  .lado.  1.  i.ot....t.  of  at  Last  P«  dagraas. 

s  r:;.i  rah?ftoMt"t;'.raT.a”,i  '.riu  romiir.;  .t..  m .  .1.,..  =..0. 

II;  ,rpS,tt?™.”S'ia"lia1rinLiS  P»»‘”*  "i  l" 

'“'Suoillt.  The  AI-EOD  gr^  compres^^='n  SUd  Si  rSuS  tSSor- 

There  are  approximately  one  million  pixels  pe  9.  page-words,  drawings,  and  the  white  space.  *- 

age  ceguireaent  aigaifioantiy.  Saster  scan  g  (SGHLl  would  ha  uaed  ia  an  iaplenentatlon  systen. 

a  ...d  that  .actot  gtaphica  aad  0P=P‘‘hf  o  -EOD  a, ala.  was  that  naata  shonid  tagtire 

ifwiiir  r.r,Si”'.rc.r“;  pm‘.m.rcro'thi:g ..... ....g *i-eo.  .......  ^..1.  opeta..on ..  t.stt..,. 


355 


,,,,,  01  the  design  and  ?h:iosophy  o:  the  ay 

jetiu  options,  no  previous  KnoUedge  oi  co  p 

ieeich.  The  P^°5ta®  ^''^"''''thfulo^n  or'dri^ce''  Select:^ ____ 

■  sel'tted  by  a  f ^  first  categorues  t  identifying  attii^.» 


.d  to  the  current  mu  options, 

IZZ  “’Vrawing  upon  the  EOb  ^^;;;‘;;",i^,".'nKno«n  ordnance. 

“’  TZ  ZZ  Zm  «.'«4se  ,  tffi  to  ..t.r  ••Vf!°irS;t  oi  istiol 

tor  that  the  EOD  technician  can  enter 

Depending  on  the  si  ■  additional  .:, ‘‘jMericai  data,  nic'naies 

"Suo, .  ;E;n--r  s.r  tsr-*  -- ■■“ 

— ;s‘^^  ..  to.  .no.  4.  -  -  -  Si:. 

,.r.U0109!  ..4  ‘”'  i°o  (  '  ..««  is  .«i  '  l74..ic.s  ....  ■<”““  “..  ’"10“  ...  ..Lots  0. 

S  mt\'. -^tscliW  ZSm  ni'.  . 

S  loUmllT,  tk'  r” ,"  J„p..s.t.  i"'  E„o  .ioM  .iii 

used  to  ...toi'  sii  40  0  lespood  to  it  01  a.S  “O”  Eiinlays  all  oi  to.  a.' 

tributes  and  their  ®  glevant  to  the  mission  (inciden  )  ^ 

".‘sr.i..‘&,,.„d  i.i...«i.4  to  .....04; i‘r...t.’‘:i=‘ii"  ;  -  srS4;“io;-irti- 


are  praser.rec 
s  isbe.a:  '. -• 


identifying 


10  aissions  entered  information  to  ®  separate  windo«  «iU,  0P®“  ^  ^isdo«  sho'is  the 

j.st..41.9  »'4‘‘  ‘'“'A  ,\„„tb  oi  8.H0‘'»“  "o  '  ,  .  ,.,  ..bucalioo.  Tkis  i.4‘ 

.m  op.. .  s‘t4MS‘o“  ^  ",n.  »Tp4«“- 

»  r.  r»»  r  r 'iUr  Srs  s  Stfr. 

►sr..s  -  “v‘ *m;”  s'iS  4/4.  ".firs  rsu  ..4  ...p...  i.4«4  'o- 

‘foSlo.  .‘S'  'n‘.r  “■*"  “  ‘  f  1..  s..i..  >•  p*‘““tio..  to. »  u 

ftf  Sfi  4- ‘^r..r.tui‘oo;r‘ ui 

rpS^shoTtV'pubU^^^^^^^^  anra'nmrU  va\r sUng  ;SSor?n°th^  candidate  list.  The 

Ift.t  a  t“4'‘“'‘ ,  ,  tte  oiipioal  papet  paOl'catio  .  j,. 

“A’=‘.?.rtC''i.°t«°“«  '«‘“iii;!AlS'o‘ptS'’op..s  a  .1.44.  tt-=‘  “4pla,s  a 


-  r.rSiSrtir.ris  ao  »-;"•  “77=^1..  1. 

r'r  "• 

publication  number,  tne 
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rnes 


‘cr.'c^  • 


-a'ar*-  n'^fictly.  ^itfirnstfiiy.  -.-c*-  - 

...cuo. .... 

-riri”. ‘s 

:vie  user  to  move  quickly  to  tHe  sectio.  ^ 

iocument  in  an  ^  the  use:  does  not  know  «hat  several  windows 

BjknoMLiatsaouJM-  ,  -jjcroact.  and  when  selected  .lU  presenv  ^  . 

known  Cate^^TrOP^i^M^°^^  'cai  be  used  in  identifying  ^.-^.f-Un  can  select  any  that 

all  of  the  known  ” 

-  rSi  SSSir^o^L-r.r.::  rs  1:1:1. 

3  The  features  Suboption  disp  y^  ^  this  n-^n  window  ontion,  unknown 

a  feature  (e.g.,  does  the  iy  any  ordnance  appears  here.  f '"Jgfggg.res^an  be  avoided. 

Kny  question  that  can  or  the  blurry  distinctions  between  categ^.-^^  Common  Name]  opens  a^ 

Category,  false  ®ts^ts  crea  e  J?hT  Ut\e  Common  names  or  aliases  which  ine 

tiord  S^^vcn  upiiouB-  ,,„,ae  tn  look  for  in  the  tine.  '>  i.„_.  cand." 

Foreign. lanquage^eras  situation  is  more  difiisul  Language  Terms 

SS'HFs'i  jrr .r:r r:i:.. 

.•  th,t  was  used  in  the  prototype  (as  modified  based  o^ 

.  ■  1  ices  the  qeneral  hardware  configuration  that  «as  ’is  g  rar,  (see  Conner  et  ai., 

„.„r,sr,'- iS  1 ...  —1  ..■  “ 

'r'SiSf’S  »n.r  -i‘YTm’  *1  ' ' 

0  CPI.  SiniBUS  386,  or  e'l'ii’® lent  JPd.  n  «  - 

math  coprocessor  is  suggested  g®  ^^5  qj  aain  memory;  16  megabytes  i®  *^5  c  *  and  yindows  3.1.  '<idile 

; 

the  neural  net  code  for  tiie  gy^rem  to  the  other.  ^  ordnance  (identification-table  ^ 

to  Bt'oie  the  otetili"!  *’'''5 ‘.f cj’«oS*p”:j«t  "““9  ‘°™'' 

:  Cs’tS:  f  - -9:,:::r::'pt-“9  >»•  *'■ »'  ^ 

p.. .:- -  =• ti” ■■“ ' 

9  Input  Device.  arruai  neurologica;  ptoc 

SoiBary  of  System  foundation  of  AI  ,  i,{o:iation  search,  retrieve.. 

Recording/Trickisg.  for  mcideur 
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information.  It  also  tracks  the  status  of  the.  incident. 

Identifying  Objects.  The  AI-EOD  system's  unique  search  and  retrieval  capabilities  assist  the  use:  ir.  ;.ent..y- 
ing  objects.  For  example,  the  search  can  be  by  characteristics,  such  as  physical  attributes,  by  common  name  c: 

alias,  by  nomenclature,  or  by  publication  number.  ,  •  .  . 

Audit  Trail.  During  a  session,  each  characteristic,  or  lack  of  a  characteristic,  is  recordec  sS  ..ne  usi:. 
enters  it.  This  record  serves  as  a  type  of  audit  trail  to  document  the  activities  of  the  user.  ^ 

Listing.  Based  on  information  the  user  provides,  the  system  displays  and  updates  a  list  of  reiev:... 
tions  in  real  time.  The  system  also  evaluates  and  rates  the  publications  with  the  best  “fit‘  at  the  top  o.  t-.e 

Viewing.  At  any  time,  the  user  can  select  for  viewing  one  or  more  publications,  consisting  of  texi  and  images, 
to  assist  in'confirming  the  identity  of  an  object.  Viewing  features  include  multiple  windows,  room-in/out,  rotate, 

and  scroll.  , 

Reporting.  When  a  hard  copy  of  the  publications  would  be  useful,  the  user  can  select  one  or  more  puOiicatiOns 

for  printing  on  a  laser  quality  printer.  The  audit  trail  notes  can  also  printed. 

AI-EOD  Features  and  Benefits  .  „  i 

A  number  of  special  benefits  and  features  associated  with  the  AI-EOD  prototype  are  summarized  in  Figure  .. 


UNII/Hindows 


eConsistent,  intuitive  iCan  conduct  parallel  iPortable 
graphical  user  inter-  searches  accessing  the 
face  (GDI)  same  documents  or  dif¬ 

ferent  documents 

•Short  learning  curve 

’  DOD  standards 


Artificial  Intelligence/  •Modeled  after  actual 
Neural  Net  neurological  proces¬ 

ses 

•Can  reach  con¬ 
clusions  based  on 
incomplete  or  inac¬ 
curate  as  well  as 
negative  data 

•Suitable  for  other 
platforms  (e.g., 
DOS/Uindows)  as 
well  as  other  applica¬ 
tions 


•Represents  an  innova-  (Combines  phonetic 
tlve  implementation  and  'fuzzy' 

of  AI  technology  searches 

•Evaluates  and  rates  (Displays  and  up- 
the  publications,  and  dates  a  list  of 


places  the  publicati- 
with  the  best  'fit' 
at  the  top  of  the  list 


relevant  publications 
in  real  time 


Image  and  Test 


Object-Oriented 

Prograuing 


•Viewing  features  in-  (Select  one  or  more  (Records  searches  as 
elude  multiple  win-  publications  for  print-  a  type  of  audit  trail 
dows,  zoom-in/out  ing  on  a  laser  printer 


•Models  reflect 
aspects  of  the  real 
world 

•Maintainable 

•Reusable 


Figure  1.  AI-EOD  Prototype  Features  and  Benefits 


•Creates  a  more  reli-  (Shorter  development 
able,  error-free  sys-  time 
tern 


•Flexible,  adaptable 
•Capable  of  evolving 


iLonger  life  span 


TEST  AND  EVALOATION 

The  test  and  evaluation  (TSE)  effort  was  conducted  to  collect  and  analyze  information  on  the  reliability  and 
validity  of  the  AI-EOD  delivery  system.  Performance  is  measured  by  time  and  accuracy  (AI-EOD  vs.  conventional). 
Demographic,  and  utility  and  usability  information  about  AI-EOD  was  also  collected. 

AI-EOD  Testing  Episodes  . 

For  AI-EOD,  performance  is  measured  by  how  quickly  and  accurately  an  EOD  technician  can  identify  an  ordnance 
shape  and  locate  the  RSP  for  the  device.  This  process  closely  relates  to  the  ordnance  identification  requirement 
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encountered  in  the  operational  environfflent.  j  ordnance  (a  shape! .  The  :e 

the  IS  number  (i.e.,  episodes  were  selected  utiliz.r.:  the  : 

a,  sel.ctl..=  ..s.  bs  .ad.  b,  Smd  bat  war.  bo: 

ice  ordnance  identification  requirements,  ^  ^  ^  ;^y  transported  from  one  test  site 

td.  tecbdlci...  .Obld  .ot  b.  a.a  ,  b  .  ^ 


thnician  must  ice 
he  reccrt i r.t  z'. 

cliOwinc  ttite- 
and  cross-serv- 
)0  familiar  that 
to  the  next.  .- 

'Table  1). 


Shape  t 
RS?  Pg. 

1/1,8 

2/4 

3/5 

4/6 


5/6 

6/8 

7/3 

8/5 


SI!  I  TEST  SHAPES 
DK  Bomb  Dnit,  8EAT,Nol  HKl  ■ 

0S  Projectile,  40m,  HHDL  8430 
BSSR  Projectile,  82m,  HEAT,  8K*881.. 

French  Landmine, APERS.Nonffietalic,  1951 

SIT  II  TEST  SHAPES 
Italian  Lindaine,  APERS,  ADPS-BRIND 
DSSR  Rocket,  '  SEAT,  PG-18 


Pnbllcation 

ID  Guide 
Page 

Inert  Sticker 

60B-3-2-11*  ^ 
60D-2-2-23-10 
60D-35-2-2 
60H-1-2-6 

11-27 

3-593 

3-297 

6-37 

NAVEODFAC  2801 
DSHC  10751 

NEODTC  0557 

301987  Inscribed 

60H“9-2"16 

60F-35-2-27 

60D-2D-3-10, 

60E-7-2-3-13 

6-66 

3-142.1 

■3-1167.2 

3-141 

•NAVSCOEOD  801095 
NEODTC  0967 

DSHC  15366 
NAVEODSCL  301804 

60T-2-2-6 

60T-2-2-1U 

60H-9-2-11S 

11-14 

11-22 

6-54 

NEODTC  0650 
Produced  Inert 
•Rubber  Trng  Aid 

.s  /.rW^’WAd  . 

i  Multiple  Ordnance  Publication. 

Table  1.  il-EOD  Practice  lad  Test  Shapes 

. . . 

D  fnr  ii  ROD  T6E  are  divided  into  six  categories:  (1)  EOD  performance  (as  de- 

Researeh  Hypotheses.  Hypotheses  for  AI  EOD  ™  ,,,  ,,noyieflge  (as  defined  by  training  time),  (3) 

fined  by  completion  time  and  errors  made  test  ep  ),  (  )  ^^^  ,9  personal  assessment  of  EOD 

EOD  experienci/ability  (as  defined  by  years  of  EOD  p  of  the  AI-EOD  (measured  by  responses  on 

si«. .-» -:rx>r‘::"r.br  s/i  "sr=  i^idTpo. 

Sr  effSs.  EachSject  was  fh^urfb^ cSSe'ShSb'^hSwas  no  specific  time  limit.  The 

Each  testing  sequence  took  ®  J  J „5i,g  ^ne  of  the  delivery  systems  and  then  used  the 

:;(S“  “7r‘:r.b:'f.b.rirt.‘ -  p-'**  ” 

<b  b.  jbbu 

ments.  The  delivery  system  was  verifie  ^ 

ingly  by  expert  EOD  technicians  assigned  to  the  ^  (instructor)  and  novice  (student) 

®  \he  system  was  then  used  to  gather  o  all  four  services  at  three  skill  levels  (as 

personnel  at  the  EOD  school.  The  *|s,.hool  students  and  EOD  assistants),  journeyman  (expenenced 

(ppppp-i  *  ■" 
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requireaents  through  asBignient  as  technical  experts,  such  as  instructors  or  technical  positions,  or  were  "laaster 

blasters')- 

Data  Collection.  NPRDC  collected  data  for  the  master  data  base  for  242  subjects  at  24  sites  between  Xarch  and 
August  1991.  The  data  base  contained  demographic  data  from  the  initial  questionnaire,  performance  data  for  each 
completed  testing  sequence,  and  critique  data  collected  from  each  subject  after  testing.  Data  were  entered  into  a 
computerized  statistical  package,  and  descriptive  statistics  were  computed. 

After  the  data  were  checked  for  completeness,  a  preliminary  evaluation  was  performed  to  inspect  the  data  for 
various  anomalies.  Based  on  this  evaluation  data  for  7  subjects  were  dropped  due  to  missing  data.  Thus,  the  final 
data  base  contained  data  from  235  subjects.  The  resultant  data  base  was  used  to  create  files  to  determine  demo¬ 
graphic  results  and  for  testing  of  the  study  hypotheses. 

Results 

Demographic  Data.  Demographic  data  for  235--10  instructors  and  11  students  at  the  Navy  EOD  school,  and  2i4 
operational  BOD  per8onnel--were  analyzed  and  are  presented  in  Table  2  . 

The  results  for  the  ALL  group  in  Table  2  might  be  summarized  as  follows:  The  235  EOD  technicians  tested  had  an 
average  age  of  29  1/2  years;  971  were  male  and  3%  were  female;  had  over  9  1/2  years  in  service  (9.6);  and  an  average 
paygrade  of  E-6  (5.8)  with  over  2  years  (2,2)  in  paygrade.  They  had  received  almost  a  year  (11.2  months)  of  train¬ 
ing  in  their  BOD  specialty;  had  been  qualified  as  EOD  technicians  for  almost  5  years  (4.9):  and  had  been  working  as 
EOD  technicians  for  over  4  1/2  years  (4.6).  Almost  89%  (88.5)  of  the  235  subjects  reported  that  they  were  currently 
working  as  EOD  technicians. 


VARIABLE  HDHBER  AID  TITLE 
(selected  variables) 


ALL 


Air 

Force 


TEST  GRODP 


Army 


29.5 

97.0 

235 

9.6 
E-5.8 

2.2 

11.2 

4.9 

4.6 
88.5% 

4.7 


1.  Age  (in  years) 

2.  Gender  (tage  of  males) 

3.  Branch  of  service  (numbers) 

4.  Time  in  service  (in  years) 

5.  Paygrade 

6.  Time  in  paygrade  (years) 

7.  Amount  or  BOD  training  (months)  , 

8.  Time  gnalifled  as  EOD  tech  (years) 

9.  Time  working  as  SOD  tech  (years 

10.  Currently  worklng*EOO  teen  (yes) 

11. Self  evahation-EOD  expertise 

12. Self  evalnation-compnter  experience  3.1 

25.  Average  time-  AI-BOD  (min.)  5.0 

26.  Average  error  rate-  AI-EOD  .04 

27.  Average  found-  AI-BOD  4.0  I 

40.  Average  time-conventional  (min.)  9.6 

41. Average  error  rate-conventional  .46 

42. Average  fonnd-conventional  3.93 

(for  variah 

5.9 

1.9 

7.4 

1.9 

1.9 

6.6 

3.4 
5.7 


ilSi 


27.7 
93.3 
60 

7.6 

4.7 
2.2 
9.0 
6.1 

5.7 
93.3 

5.2 

(for^variable  11 

(for  variable  12 

^;03  (.751) 

I  4-0 


11.5%) 

98%) 


.60  (15%) 
3.92 


Marine 
Corps 

(average  responses) 

28.9 
94.4 
36 
8.5 
6.1 

1.7 
10.8 
4.9 

4.8 

100.0 

5.3 

O^none 
3.1 

O^none 

4.7 
.03 

4.0 

7.8 

.42  (10.5%) 


;.75%) 


3.92 


29.9 

100.0 

52 

10.2 

5.9 

2.2 

7.6 

4.3 

4.3 

96.2 

4.5 

9:ex^ert) 

9sex|ert) 

*’.06 
4.0 

9.5 
.33 

3.92 


Mavy 


30.9 

98.9 
87 
11.0 

6.3 
2.5 
15.0 

4.4 
3.9 

75.9 
4.4 

3.1 


(1.5%) 

(8.25%) 


5.9 

.04  (1%) 
4.0 

10.9 
.46  (9 

3.97 


25%) 


59.1  like  the  current  system 

60.1  like  the  new  AI-EOtl  systu 

61.1  wonld  rather  have  the  current 
system  than  the  AI-EOD 

62.1  wonld  use  the  AI-BOD  systu 
rather  thu  the  current  systu 

63.  The  AI-BOD  was  easy  to  use 

64.  The  AI-BOD  needs  major  modification 
to  be  useful 

65.1  U  fully  qualified  in  BOD 

66.1  u  fully  qualified  in  computers 


es  59-66  isstrongly  agree  --  through  —  9=strongly  disagree) 


6.0 

1.7 
7.3 

1.8 

1.8 

6.7 

2.8 

5.5 


5.6 

2.1 

7.2 

2.0 

1.9 

6.7 

2.9 

5.3 


5.1 

2.1 

7.1 

2.4 

2.3 

6.5 

3.3 
5.9 


6.5 

1.9 

7.7 

1.7 

1.8 

6.7 

4.1 

5.9 


TABLE  2.  Duographic  Results:  Selected  variables 

In  terms  of  their  expertise  or  knowledge,  the  subjects  considered  that  they  had  from  some-to-much  expertise  in 
EOD  and  they  reported  little-to-some  knowledge  and  experience  with  computers. 

The  results  of  the  testing  on  the  eight  test  episodes  (four  AI-BOD,  four  conventional)  show  that: 

Using  the  AI-EOD  system  the  subjects  took  an  average  of  5  minutes  to  identify  the  unknown  shape  and  determine 
the  RSP  for  the  four  test  episodes,  with  .04  identification  errors  per  test  exercise,  which  is  an  error  rate  of  1%; 
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they  correctly  identified  all  four  shapes  of  the  test  (1001  solution  me)^  __  , 

Using  the  conventional  approach  (i.e.,  docusants  "355  of  45  g^ors  oer  test  seccence.  * 

sinutes  (9.6)  to  identify  the  shape  and  deteriine  its  RSP,  ^.th  .nd  ^ 

"  ".he^sM^ol  ii^5in!onr;;on  co^;^V 

care  for  the  current  (conventional)  system;  they  liked  the  J.,  ..=j  -.oe 

Slysief  S^y  smaLsr^ 

second  query  about  their  ^00  and  computer  qua  i  1  ,  Y  results  for  each  service  : 


ECO 


conventional  (AI-EOB  tiirie 


anrthey  moderately  disagreed  that  they  were  qualified  in  the  of  rs. 

be  reported  in  a  similar  fashion  as  was  done  for  the  entire  popu  ation  (ALL). 

Hypotheses.  Results  of  hypotheses  testing  are  presented  below. 

u  “irSpl.t.  test  eurcl^s  fitei  .»«!  *1-™  tl.a.  ..l.s 

”■  Tb”«riili"l..  f».r  .rro..  .ith  .I-EOD  tha.  .itb  c»..a.,l..al  (M-EO.  art.ia  .a,  ccaatlaa- 

al  errors).  rat-o  ncim  ii-EQD  than  when  using  conventional  (AI- 

Subjects  will  have  a  significantly  higher  identification  rate  using  Ai  eod  than  g 

EOD  identification  rate  vs.  conventional  cinnificantlv  shorter  using  the  AI-EOD  system 

systeii  CMpareJ  to  conveitloial  leaii  to  ‘  ,,  p„s.,ct”e  tit  fact  that  the  lasolts  -aeie 


Factor 

Average  Time  of  Completion  (in  minutes) 
Avenge  Error  Rate  Inusber/test  set) 
Average  Kaaber  Fouad  (per  test  set) 


PERFORMBCl  COMPARISOHS 
Coapnter  Conventionil 
5  043  9  .  4 

.030  .460 

4.000  3.936 


Average  Response  to  'I  LUe' 
Average  Response  to  'I  Roald  Hsi 
»p  <  .05.  ,  ,  , 

ssResponses  range  from  l=Strongiy 


1.902«a  5.898SS 

1.949ii<i  7.387H 


F  Ratio  Probabil  ty 
235  232.692  1.800E-136 
235  53:912  8.B00E-1  a 
235  6.992  8.462E-03is 


235 

235 


613.836  4.000E-136 

1020.830  7.000E-149 


Agree  to  9=Sttongly  Disagree. 


3.  Coiparlson  of  Perforaance  and  Prefereaei:  AMOD  i  Conventional  Systeis 


,10  .ollabiUty  oi  tils  m.  tl.  loot 

(Lin;.  ;:\;irt'::i.“ifaLat'ls;“:^ 

also  have  major  impact  on  manpower,  personnel,  and  training  decisions. 

,,  JS-” rrrr  =,=.:=  =,•= rt 

is  so.o.lan^H.  Ills  Itti*  to  a  ...Ml  o  co.  Its  o  '”'‘*‘,“^1  ™-.  acco.p'.iate'-  ;t  a 

Li;  iL';;;..  n.  taai'Mi.,  .tcpusiap ..  laa.,  n 

.Lrlwortall.  0!  tlaa.  »tUsl..  is  tlat  tl.  ‘-(tor  » 

,1,  EOD  ttclaiclais  cat.ot  as.  tl.  t.ctoical  l.foi.atio.  l.atn.a  at  .clool.  lit  EOD  sc-.o.  ptotioe. 
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about  ordnance  characteristics  and  how  to  identify  and  deal  with  each  type  of  ordnance  »ii.h  .hese  Lha-.^c^eris^..^ 
However,  the  conventional  identification  procedures  restrict  the  EOD  technicians  to  only  two  .hara..  .o...^ 
initial  identification  process,  major  body  diameter  and  overall  length.  They  select  ^  category  .  o--..an.t,^...... 

through  a  series  of  pictures  until  they  identify  the  device  using  the  diameter  and  ieng.h.  ..  ..he  de...e  .=  ....  ... 


inroaQr.  a  senes  ui  uilluics  uu;.ii  wucj  - - - w  ^  *.•« 

th.  iii«  ,  th.,  Btlect  .  ..as-: 


Llie  Iirsl  LdtCUUiJi  ocxcwt  uuwwuv^  - - - J  ^  i.L.fc 

limits  technicians  to  performing  three  basic  tasks,  only  one  of  which  is  moderateiy  di  uu...  w. 

diameter,  measuring  length,  and  selecting  category  of  ordnance. 


FACTOR 


Perfdiiance  Categories 
jil-EOD  Conventional 

TIKE  ERRORS  TIME  ERRORS  FOUND 


.02108 

.05379 

.08186 

.07804 

.11296* 

.12820* 

.08567 

.15243* 

.07967 

.01901 

.09592 

.00079 

.21611* 


-.02933 

.02922 

.06231 

.05747 

.02674 

-.04092 

-.00235 

-.00358 

.06158 

.03497 

.07578 

.09782 

.00237 


.09454 

.18244* 

.03561 

.02910 

.07629 

.13730* 

.14465* 

.09049 

.21366* 

.00869 

.10506 

.10473 

.19320* 


.08385 

.12623* 

.17106* 

.14756* 

■.19531* 

.22369* 

-.16388* 

.18588* 

-.19861* 

-.08134 

-.10831* 

.04334 

-.14944* 


.03100  235 

.07868  235 


.05362  235 
.04384  235 
.01034  235 
.11071*  235 
■.01319  235 
.10214  235 
.06719  235 
-.03029  235 
.05946  235 
-.02905  235 
.04613  235 


EOD  Training 
Time  in  Service 
Time  EOD  Qualified 
Time  as  EOD  Tech 
EOD  Experience 
EOD  Qnalification 
Computer  Experience 
Computer  Qualification 
Age 

Gender 
Fajgrade 

Time  in  Paygrade 
Currently  lorking 

*p<.05  «  X 

Table  4.  Correlation  Matrix  for  AI-EOD:  Hypotheses  Factors  vs.  Performance 

AI-EOD  allows  for  more  ordnance  knowledge  as  technicians  answer  the  questions  about  the  unknown  device.  Train¬ 
ing  does  not  correlate  with  AI-EOD  because  of  the  nature  and  design  of  the  system,  the  neural  net  will  accept  a 
nuLr  of  erroneous  inputs  and  still  assist  the  technician  in  correct  identification  of  the  device.  The  appare  . 
lack  of  relationship  of  knowledge  (training)  to  job  performance  should  be  investigated  to  ensure  that  the  cause 
not  one  of  relevance,  and  to  ensure  the  AI-EOD  system  is  valid  and  reliable  as  a  JPA. 

S^b1ectfwhfhavfhigher^xperlence/abllity  as  measured  by  time  in  service,  time  BOD  qualified,  time  working  . 
as  EOD  technician,  EOD  experience  (self-report),  and  EOD  qualification 

better  as  measured  by  AI-EOD  time,  AI-EOD  errors,  AI-EOD  identification  rate,  conventional  time,  conventional  er 

rors,  and  conventional  identification  rate  (EOD  experience/ability  vs.  performance). 

Results.  Table  4  shows  no  general  relationship  between  performance  factors  and  experience  when  using  the 
AI-EOD  sYSteTlexcept  for  the  two  factors  of  self-reported  EOD  experience  and  qualification,  and  time  .  /here  was, 
however, ^  of  general  and  consistent  relationship  between  performance  and  experience/ability  (particularly  with  the 

nerforaance  factor  of  'number  of  errors')  when  using  the  conventional  approach.  ,  . 

Discnssion.  This  unexpected  result  could  very  well  be  associated  with  the  task  accomplishment  ^estnctiM 
encountered  with  the  EOD  training  hypothesis:  that  is,  the  conventional  system  does  not  allow  for  special  perform¬ 
ance  as  a  result  of  greater  ability  and/or  experience  and  the  AI-EOD  system  tends  to  support  users 
skills  and  abilities.  Consequently,  the  relationship  is  weak,  at  best,  between  performance  and  experience/abi  iiy. 
This  lack  of  relationship  should  be  investigated  to  ensure  that  the  cause  is  not  one  of  training  relevance,  and  .o 
ensure  the  AI-EOD  system  is  valid  and  reliable  as  a  JPA. 

A’sipificMtllTa'rSSflitSir  of  subjects  will  prefer  the  AI-EOD  delivery  system  to  the  conventional  approach 

as  measured  by  their  responses  to  the  following  critique  questions: 

a.  I  like  the  current  microfiche  system  for  identifying  ordnance, 
b  I  like  the  new  AI-EOD  computer  system  for  identifying  ordnance. 

Responses  ranged  from  l=strongly  agree  to  9=strongly  disagree  ('Like'  conventional  vs. 

Two  other  critique  questions  of  preference  were  asked: 

c.  Given  the  choice,  I  would  rather  have  the  microfiche  system  than  the  AI-EOD  system, 
d  If  both  systems  were  available,  I  would  use  the  AI-EOD  system  rather  than  microfiche.  ^ 

Responses  ranged  from  l=strongly  agree  to  9  strongly  disagree  ('Dse'  conventiona  vs  .se^  A  /uu). 

Results.  Table  3  shows  a  general  and  consistently  significant  preference  for  using  the  AI-EO.  system  »5 


ike'  AI-EOD 
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loffipared  to  using  the  conventional  systes.  ^  c  , 

Discussion.  The  responses  to  the  tBO  preference  questions  revealed  a  very  strong  and  consistent  preter- 

en^'e  fo’’  the  AI-EOD  systei  over  the  conventional  system.  -Inspection  of  the  open-ended  questionnaire  and  resu..s  ; 
comments  by  subjects,  leaves  little  doubt  that  the  subjects  strongly  preferred  the  AI-EC: 
that  the  computer  literacy  and  acceptance  of  computers  by  the  average  EOD  technician  are  3igr.i.;:ar.t.y  aes.e. 
the  anecdotal  inputs  provided  at  the  start  of  the  project.  The  high  level  of  user  acceptance  and  preference  c.e=r 
indicates  that  the  EOD  operational  and  training  communities  would  welcome  the  computerized  approach,  .mplemen.c.., 
of  the  AI-EOD  system  with  its  artificial  intelligence  component  should  be  accelerated. 

5.  Coipster  Experience/QBillficitiOB. 

There  will  be  no  significant  relationship  between  computer  experience  or  qualification,  self-reported  on  com¬ 
puter  knowledge/eiperience  (using  a  scale  from  0=none  to  9=expert)  and  ’I  consider  myself  fully  qualified  ^e 
of  computer"  (responses  ranged  from  l=strongly  agree  to  9=strongly  disagree),  and  performance  results  as  measurd  by 
AI-EOD  time,  AI-EOD  errors,  AI-EOD  identification  rate,  conventional  time,  conventional  errors,  and  convenfiona. 

identification  rate  (computer  eiperience/qualification  vs,  performance).  ,  ,  , 

Results.  Table  4  shows  no  general  and  consistent  relationship  between  computer  experience  and  pertor.i;- 

ance,  except  for  a  minor  relationship  to  the  conventional  system.  j 

Discussion.  Computer  literacy,  use,  and  acceptance  by  the  EOD  technicians  who  took  part  in  this  study  were 
significantly  greater  than  expected  at  the  start  of  the  project.  The  lack  of  significant  relationship  between 
reported  computer  capabilities  and  performance  confirms  that  computer  expertise  is  not  necessary  to  utilize  the^Ai- 
EOD  delivery  system.  Further  investigation  is  needed  to  ensure  that  the  results  of  the  computer  vs.  performance 

results  and  conclusions  are  valid;  that  is,  the  construct  of  the  user  interface  of  the  AI-EOD  system  can  accommo¬ 

date  personnel  with  little  or  no  computer  background. 

There  will  be  no  significant  relationship  between  general  factors,  as  measured  by  responses  to  questions  ad¬ 
dressing  age,  gender,  branch  of  service,  and  paygrade,  and  performance  results  as  measured  by  AI-EOD  Ume,  AI-EOD 
errors,  AI-EOD  identification  rate,  conventional  time,  conventional  errors,  and  conventional  identification  rat 

(general  factors  vs-,  performance).  ^ 

Results.  Table  4  shoos  no  general  and  consistently  significant  relationship  between  the  general  factors 

and  performance,  using  the  AI-EOD  or  conventional  system. 

Dlscnsslon.  The  design  of  the  AI-EOD  delivery  system  interface  and  search  techniques  successfully  accom¬ 
modate  the  differences  presented  In  each  of  the  general  factors.  Further,  the  lack  of  any  relationship  between  any 

of  the  general  factors  and  performance  measurements  appears  to  further  confirm  the  limiting  aspects  of  the  conven¬ 

tional  identification  system  as  was  suggested  by  the  training  hypothesis.  The  conventional  system  does  not  permi 
any  variation  in  approach  resulting  from  differing  abilities  or  general  factors.  The  lack  of  relationship  O;  h 
general  factors  and  performance  needs  further  investigation  to  ensure  the  validity  and  reliability  o.  the  AI-EO. 
JPA.  This  would  further  support  or  refute  the  accuracy  of  previous  conclusions  regarding  AI-EOD. 

GEHEEAL  COSCLOSIOEE 

The  AI-EOD  delivery  system  proved  to  be  a  reliable,  valid,  and  user  friendly  job  performance  aid  that  signifi¬ 
cantly  improved  the  performance  of  EOD  technicians  at  all  skill  levels  and  across  all  services.  EOD  technicians 
participating  in  the  test  and  evaluation  study  wanted  to  have  this  new  tool  as  soon  as  possible.  As  a  resuL  the 
AI-EOD  system  should  be  moved  to  the  implementation  phase. 

FOTDRE  EFFORTS  .  .  ^  , 

During  implementation  the  AI-EOD  system  should  be  further  tested,  to  confirm  the  validity  and  reliability  of 
the  approach.  Also,  the  use  of  neural  networks  should  be  investigated  for  occupational  fields  (e.g.,  maintenance, 
intelligence,  accounting,  supply,  tactics)  and  other  data  base  types.  This  new  type  of  tool  (JPA)  will  require 
considerably  more  development,  refinement,  and  investigation  for  optimum  use. 


Conner,  H.B.,  Madrid,  R.,  ailliams,  R.A.,  i  Holland,  J.  (1992,  January).  Artificial  Intelligence  -  Explosive  S:.L 
nance  Disposal  Information  Search/ Retrieval,  and  Delivery  System  (in  press).  San  Diego:  Navy  Personnel  Researc..  a..u 

Development  Center. 
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•  REDUCE  CAREER  AND  SCHOOL  ATTRITION 

•  ENHANCE  guard/reserve  EOD  TECHNICAL  SKILLS 
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HIGHLY  REALISTIC  MICROCOMPUTER-BASED  TRAINERS  AS  AN 
ALTERNATIVE  TO  TRADITIONAL  SIMULATION 

Dr.  Richard  W.  Davis 
Dr.  Anthony  C.  Phelan 


Regency  Systems,  Inc. 
Champaign,  IL 


Paper  presented  at  the  Third  Annual  Airborne  Weapons  Technology  Review 
and  Training  Exposition:  San  Diego,  CA,  January  16,  1992 


ABSTRACT 

This  presentation  surveys  the  benefits  of  microprocessor-b^ed  trainers.  Micro-trainers, 
an  emerging  class  of  part  task  and  whole  task  trainers,  exhibit  most  of  the  benefits  of 
more  complex  simulators,  but  at  much  lower  cost.  The  new  trainers  exhibit  improved 
flexibility  and  ILS  characteristics.  Using  current  and  near-fuhire  computer  technology,  it 
is  possible  to  build  highly  realistic  look-alike/work  alike  devices  to  replace  actual 
aircraft  or  weapon  system  hardware  and  software  in  training  systems.  A  key  feature  of 
the  new  micro-based  trainers  that  separates  them  from  previous  generations  of  part  task 
trainers  is  the  fidelity  they  achieve  in  emulating  the  man-machine  interface  of  modern 
aircraft  and  weapon  systems. 

Costs.  Of  the  several  benefits  to  training  discussed  in  the  presentation,  cost 
reduction  is  the  principal  advantage  of  micro-trainers  over  simulators.  The  authors 
examine  the  sources  of  costs  in  simulators  and  trainers  built  from  actual  equipment  to 
gain  insight  into  why  they  cost  so  much.  The  main  source  of  traditional  training 
simulator  cost  is  shown  to  be  from  the  increased  use  of  computerized  electronics— 
especially  the  avionics-in  modem  aircraft  and  weapon  systems.  The  cost  of  displays  and 
the  user  interface  are  shown  to  be  modest.  At  the  heart  of  the  micro-based  approach  is 
the  idea  of  using  a  computer  to  mimic  a  computer. 

This  observation  about  cost  leads  to  a  strong  argument  in  favor  of  using  the  nucro- 
based  approach.  As  electronics  in  the  systems  grow  more  complex  and  more  integrated, 
the  general  view  is  that  these  computerized  electronics  are  harder  and  harder  to 
emulate,  so  the  cost  of  emulating  the  system  would  approach  the  cost  of  using  the  actual 
system  electronics  in  many  cases.  This  paper  refutes  the  argument  that  emulating 
modern  systems  is  more  difficult  than  emulating  older  systems.  Examples  of  micro- 
based  trainers  are  introduced  in  the  presentation  to  show  that  low  cost  emulation  of 
the  man-machine  interface  and  "sub-systems"  can  be  accomplished  at  moderate  cost. 
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Realism  and  Training  Fidelity.  Even  professional  trainers  succumb  to  the 
argument  that  using  the  actual  aircraft  hardware  is  important  because,  "there  is  no 
substitute  for  the  real  thing."  In  electronics  displays,  however,  the  real  thing  may  be  a 
CRT  display.  CRTs  and  keypad  input  devices  at  the  man-machine  interface  lend 
themselves  well  to  low  cost  emulation.  The  micro-based  trainers  are  remarkably 
accurate  replicas  of  the  actual  equipment— often  good  enough  to  fool  the  subject  matter 
expert.  In  addition,  the  increased  flexibility,  feedback  analysis,  and  control  possible  with 
programmable  trainers  provide  training  benefits  not  generally  available  in  an  actual 
hardware  based  trainer. 

Case  Examples.  Four  examples  of  advanced  emulators  are  reviewed  to  illustrate 
the  main  points  of  the  presentation.  The  examples  range  from  small  emulators  used  as 
part-task  training  aids,  to  large,  integrated  systems  that  perform  at  levels  comparable  to 
traditional  simulators.  The  cases  demonstrated  are: 

•  A  low  cost  cockpit  Control/Display  Unit  (CDU)  typical  of  modern  glass 
cockpit  avionic  keyboard  and  display  devices. 

•  A  portable  aircrew  trainer  (PAT)  for  the  P3C  Orion  Tactical  Coordinator 
(TACCO)  position. 

•  An  integrated  multimedia  trainer  for  the  AN\UYQ-21  shipboard  command 
and  control  console. 

•  A  "Cockpit  Autoflight  Procedures  Trainer"  capable  of  emulating  all  of  the 
flight  deck  displays  of  a  modern  transport  aircraft. 

Using  these  examples  of  current  emulation  technology,  the  presenters  will  support  the 
following  points  in  their  presentation: 

•  Even  complex  systems  can  be  emulated  with  extremely  high  degrees  of 
realism. 

•  Selective  use  of  critical  parts  of  the  real  man-machine  interface  can  greatly 
increase  realism  and  user  acceptance. 

•  Cost  reductions  of  75%  are  routinely  achievable  over  actual  aircraft 
systems. 

•  Unencumbered  from  actual  aircraft  support  systems,  the  trainers  offer 
increased  portability  and  operational  flexibility. 

•  Significant  training  benefits  result  because  of  increased  control  and 
interface  to  other  training  technologies  such  as  multimedia  systems. 
Support  issues,  including  tracking  changes  to  the  actual  system,  are  fully 
addressed  with  well-designed  emulators. 

•  Low  cost  commercial  software  can  be  used  to  program  and  control  these 
trainers. 

The  summary  includes  a  brief  discussion  of  the  key  points.  In  addition  the  authors  have 
provided  a  set  of  notes  for  contracting  officers  and  others  involved  in  preparing 
solicitations. 


Thirty  years  ago  aircrew  training  got  done  using  a  combination  of  three  basic 
approaches:  For  about  $4,000  to  $5,000  per  hour  of  direct  cost,  the  aircrew  could 
operate  the  actual  aircraft  on  a  training  mission.  For  about  $300  to  $1,000  per  hour,  the 
aircrew  could  operate  a  simulator.  For  under  $100  per  hour  a  variety  of  classroom  or 
minimally  functional  mockups  and  part  task  ^rainers  were  available.  These  often 
included  the  so  called  "paper  tiger"  or  primitive  part  task  trainer. 

Today  the  training  need  and  the  economics  have  changed  radically.  The  cost  of 
operations  with  the  tactical  systems  have  increased  tremendously,  as  have  the  cost  of  the 
simulators.  As  an  example  of  this  trend,  the  Boeing  Commercial  Airplane  Company 
recently  purchased  a  cockpit  simulator  for  its  747-400  long  range  commercial  aircraft. 
The  $18  Million  cost  of  the  simulator  is  about  the  same  price  as  the  cost  of  an  actual 
707  aircraft  thirty  years  ago. 

The  combined  effect  of  increased  training  need  and  increased  simulator  and  operational 
aircraft  cost  has  combined  to  increase  the  demand  for  simulator  and  aircraft  time  while 
making  that  time  much  more  expensive. 

At  a  recent  symposium  on  emulation  at  the  Naval  Ordinance  Station,  Indian  Head,  a 
Navy  spokesperson  called  for  a  re-definition  of  emulation.  The  classical  usage,  they 
pointed  out,  referred  to  the  imitation  of  one  computer  system  by  another.  But  the 
applied  usage  for  training  and  other  related  disciplines  they  defined  as: 


low  cost  device  that  replicates  the  essentia!  functional  and  physical 
characteristics  of  Its  high-cost  tactical  counterpart  in  a  non-tactica!  environment. 

Both  concepts  of  emulation  are  important.  The  training-oriented  definition  applies  to 
micro-trainers  as  the  term  is  used  in  this  paper.  But  the  technology  of  the  micro-trainer 
often  involves  using  one  computer  to  imitate  another  as  the  more  traditional  definition 
implies. 


MICRO-TRAINERS;  DEFINING  THE  NEW  CLASS  OF  DEVICES 

To  address  this  growing  need  for  training  emulators  that  approach  simulators  in  their 
ability  to  train,  but  at  costs  much  below  those  of  conventional  cockpit  or  full  system 
simulators,  several  important  training  projects  have  turned  to  a  new  class  of  device. 

This  new  class  of  system,  dubbed  here  the  micro-trainer,  is  demonstrating  its  ability  to  fill 
a  broad  spectrum  of  training  needs  include  both  traditional  simulators  and  traditional 
part  task  trainers.  The  term  micro-trainer  is  introduced  here  to  emphasize  the 
application  of  micro  electronics  in  the  trainer;  a  more  general  term  that  could  be  used 
equally  well  is  emulator. 

The  essential  characteristics  of  a  micro-trainer  include  these  items: 
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•  The  training  device  uses  an  internal  microprocessor  or  rnicro-controller 
with  an  operating  system  for  interface  and  control  functions. 

•  The  training  device  physically  resembles  or  may  precisely  copy  a  piece  of 

tactical  equipment  or  an  operational  system.  r  ,  j  •  • 

•  The  training  device  performs  some  or  all  of  the  functions  of  the  device  it 

emulates  as  required  by  the  training  program. 

•  The  device  is  software  programmable,  and  may  support  some  emulated 
functions  separate  from  any  other  computer. 

•  The  device  contains  standard  communications  interfaces,  which  permit  it  to 
be  integrated  into  a  larger  trainer,  or  to  be  used  as  part  of  a  CBT  or  other 
interactive  instructional  program. 

•  The  trainer  contains  self-diagnostic  and  test  routines  to  assist  m 
maintenance  and  support. 

•  The  device  has  a  development  environment  or  programming  support 
environment  for  applications  development. 

•  The  unit  MAY  support  the  communications  interface  of  the  operational 
system,  and  could  be  used  to  replace  the  operational  device  in  a  simulator 

or  training  system. 

While  these  requirements  may  sound  simple  enough,  devices  that  meet  all  of  the 
requirements  have  been  introduced  only  in  the  past  few  years.  To  dramatize  the  key 
differences,  it  is  useful  to  compare  a  micro-trainer  with  an  older  part  task  trainer  that 
used  electronics.  An  example  of  an  earlier  generation  part  task  trainer  will  help  to 
highlight  the  differences.  That  earlier  trainer,  for  a  commercial  transport,  employed 
digital  logic  in  the  trainer.  But  following  typical  practice  of  five  to  ten  years  ago,  the 
electrical  connections  for  the  lights  and  switches  were  individually  routed  back  to  a  mim 
computer  for  control.  In  a  micro  trainer  the  control  micro-processor  would  be  built  into 
the  trainer  itself,  and  only  a  serial  or  parallel  control  bus  would  connect  the  trainer  to 
the  computer.  In  the  earlier  trainer  the  electrical  connections  were  all  hard-wired  to  the 
computer  Not  only  do  these  hardware  connections  limit  the  flexibility  and  control  of 
the  trainer,  but  they  cause  an  ILS  headache  as  well.  The  hardwired  connections  prevent 
the  trainer  from  being  portable,  since  even  if  the  trainer  and  computer  can  be  moved, 
the  cost  and  time  required  to  disconnect  and  reconnect  the  wiring  is  prohibitive. 

Finally  there  is  no  application  development  environment  for  the  older  trainer.  The 
primitive  computer  I/O  function  means  that  a  line  of  code  was  written  to  control  each 
device  and  that  each  simulated  knob,  light,  or  switch  on  the  trainer  was  supported  in  a 
seaparate  software  routine.  The  older  style  trainer  was  supported  entirely  by  custom 
written  software  written  in  assembly  language  or  FORTRAN. 

The  micro-trainer  represents  advances  on  many  fronts.  In  the  micro-trainer,  code  is 
written  in  C  or  C+  +.  While  ADA  support  is  available,  ADA  adds  significantly  to  the 
cost.  The  presence  of  an  internal  commercial  operating  system  in  the  trainer  makes  it 
possible  to  write  code  in  a  full  featured,  productive  development  environment,  and  even 
to  simulate  the  performance  of  the  trainer.  This  idea  of  a  programmable  trainer  with 
extensive  software  control  and  a  powerful  external  development  environment  is  probaWy 
the  biggest  factor  in  the  emergence  of  the  micro- trainer  as  a  distinct  type  of  device,  ine 
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progrsmniability  is  irnportsnt  because  it  permits  precise  control  over  the  training  device- 
-indirectly  contributing  to  increased  realism  and  reduced  cost. 


Figure  1.  Conipsrisoii  of  Simulaitor  End  Micro-XrEiner 
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The  Micre-Trainer  as  Computer  Peripheral:  Improving  Training 


The  argument  put  ‘  specifically  in 

Tm^^i^overfilrB- "  ainer  does  provide  real  improvements  that  the 
trainer  can  see  and  use 


one  convenient  way  to  start  thinhing  about 

K^aV"  It  poU'pich  any  Keypad  and  display  from  a  man- 

machine  interface. 


lliciviiixxw 

With  such  a  peripheral,  the  “““"B  ‘‘‘oj°’JfoLTCTStem'are  twSle  'the  trainers  can 

All  of  the  task  specific  cues  from  he  opera  lonaijstem  are 

now  show  the  learner  a  highly  "  d^P'^Vo^ 

response  device  in  an  interactive  S  peripheral  micro-trainer  an 

examples  discussed  below,  it  wi  e  p  .  ..  •  from  the  real  thing  as  it  is  used 

exact  replica  of  the  operational  system--  .fainers  even  a  few  years  ago,  is  that  the 
in  training.  The  new  feature  not  ®  I  hoTmultimedia  compter.  The  training 

micro-trainer  is  fully  under  the  contro  o  displays  to  the  learner,  and  can 

developer  can  precisely  control  w  a  makes  The  level  of  training  control 

determine  exactly  what  response  the  learner  makes. 

introduced  by  the  micro-trainer  is  remarkable. 


Realism  is  Important.  The  and  avionics.  As  the  units  get  smaller 

cost  trainers  is  a  byproduct  of  modern  look-alike/work-alike  replicas  of  the 

and  more  modular,  it  is  ^^Uy  ,  switches,  and  computer  displays  that 

man-machine  interface,  '^e  taobs,  available  Even  when  they 

show  up  in  aircraft  or  on-bo^d  ship,  aSance  and  function  can  be 

are  customized  for  a  parti^lur  can  be  emulated 

mimicked  very  precisely.  Thus,  ^ogrammability  of  the  trainer  helps  the 

with  extrernely  high  degrees  of  reahs  .  ^niatched,  even  when  different  lights 

realism.  Tite  blink  rate  of  lights  throw  and  a  lamp  illuminating  can  be  controlled 

are  used.  The  latency  between  a  r  material.  The  delay 

without  the  requirement  of  wnting  speci  complex  issues  like  the 

mov^oTmet^^^  of  annunciators  can  be  made  more  real  because  of 

the  precise  control  available. 


Selective  Realism.  Achieving  high  sophisticated 

important  goal  in  training  design  as  learn  mme  a 

learners  are  in  using  the  cues  in  simulators,  or  using  the  actual 

need  for  ■'a®’™  That  level  of  realism  can  now  be  achieved  in 

r  rta;r"lerice:.  mre  is  a  substitute  for  the  real  thing. 
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A  hallmark  of  the  trainers  built  at  Regency  Systems,  Inc.  is  the  selective  use  of  critical 
parts  from  the  real  man-machine  interface  to  greatly  increase  realism  and  user 
acceptance.  As  an  example,  in  our  cockpit  trainers  for  commercial  aircraft,  all  of  the 
panel  knobs  are  "actual  aircraft"  hardware  with  the  same  part  numbers  as  those  found  in 
the  manufacturer’s  parts  lists.  Many  of  the  electrical  switches  and  lights  are  also  real  . 
But  the  avionics-the  electronics  modules-are  emulated. 

Often  the  decision  to  use  the  actual  device  or  a  lower  cost  alternative  is  made  on  a 
device  by  device  basis  with  the  customer.  The  trade-off  is  among  these  factors: 

»  The  accura^  of  the  task  cues  provided  to  the  student. 

®  The  overall  appearance  and  contribution  to  a  feeling  of  "realism". 

»  The  reliability  of  the  device  in  the  training  application. 

«  The  cost  of  the  various  approaches. 

Realistic  Displays.  The  most  critical  aspect  of  the  micro-trainer  in  generating 
realism  is  the  syswtem  interface— the  part  of  the  trainer  that  the  student  touches  and 
sees.  Realistic  emulation  of  the  tactical  system  display  requires  special  attention  in  both 
hardware  and  software.  In  some  cases  it  is  possible  to  obtain  the  tactical  equipment 
display  and  integrate  it  into  the  micro  trainer.  This  is  especially  true  if  the  display 
consists  of  commercial  projection  switches  or  devices  other  than  CRTs,  To  add  realism 
to  it  trainers,  Regency  Systems  has  produced  custom  LCD  displays  and  has  used  actual 
tactical  switches  to  produce  high-accuracy  micro-trainers.  There  are  many  "tricks  of  the 
trade"  involved  in  making  low  cost  trainers  look  and  feel  like  the  real  thing. 

The  greatest  technical  challenge  to  achieving  realism  is  in  CRT  displays.  Most  cockpit 
displays  in  use  today  are  calligraphic,  or  "STOKER"  displays.  This  CRT  technology  is 
employed  because  of  its  high  brightness,  and  because  several  generations  of  engineers 
have  perfected  the  electronics  needed  to  make  stoker  displays  work  in  high  performance 
aircraft  under  high  G-forces,  and  at  high  altitude.  But  these  devices  are  probably  the 
most  expensive  part  of  the  avionics  system,  and  therefore  are  the  first  things  to  be 
critically  examined  in  a  cost  reduced  trainer. 

Only  recently  have  commercial  computer  electronics  and  graphics  controllers  gotten  fast 
enough  to  closely  approximate  the  look  and  performance  of  a  stoker  display.  Even  now, 
precise  emulation  of  stoker  displays  is  achieved  only  with  a  combination  of  outstanding 
graphics  hardware,  and  specialized  software  to  give  the  display  the  right  look.  Two  of 
the  examples  discussed  in  this  paper  use  raster  emulation  of  stoker  displays.  One 
example  is  from  cockpit  avionics-the  other  is  from  a  shipboard  weapon  system.  Both 
efforts  are  good  examples  of  selective  realism,  and  both  contribute  significantly  to  cost 
reduction. 

Cost  Reduction.  Several  factors  combine  to  give  the  micro-trainer  a  significant 
economic  advantage  over  actual  equipment-based  trainers  and  simulators.  Among  these 
factors  are: 
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.  Commercia.-off-.he-she^  (COTS)  electronics  are  used-M.L  SPE 

S:  c— :  and  funcrions  no.  needed  for  .raining  can  be 

.  CScnve.  commercial  software  .oois  can  be  used  .o  heip  lower 

sUS  in  hardware  aione  are  possible  by  elimina.ing  espens.ve 
"fte"rairwor.hiness  or  orber  .es.s  recnired  for 

•  IJbor  orThUard  tactical  eqmpmenr. 

While  it  is  difficult  to  generalize  an^Suate^^^  {’f‘ne"f"”’ 

built  for  apptotatha  a  y  ,  micro-trainer  costtng  less  than 

raSss:i^=:ost. 

Portability.  Unencumbered  from  a«“J,“;"'"MicroSec^^^^^^^  generally  permit 

offer  increased%rtability  and  operational  instruments  and 

the  electronics  for  a  niicro-trainer  to  be  s  j^cDU  for  commercial  transports  is 

avionics  modules.  The  a\igh  resolution  graphics  tSt  is  the 

useful.  The  entire  framer  in  a  small  box  that 

hard  disk,  and  a  P'^^fai^ 

same  physical  size  as  th  <•  v.  RS-232 

This  pormbility  is  an  ™P"^SCn"oftcn 

,  ,,„3,hng 

Wnce  ro  Other  f  ^ 

from  con^oj  of  PCs  a ‘S-trainer  as 

^r'a^rare  wen  seethe potential  benefits  of  using  the 

^re"  ani  rmu— 


386 


interactive  video,  conventional  CBT,  or  intelligent  part-task  trainer  in  a  single  system. 

In  the  training  for  a  particular  avionics  system,  the  learners  first  see  a  brief  video 
segment  demonstrating  a  particular  function.  Then  they  work  through  a  lesson  on  the 
PC  with  an  animated  graphic  of  the  system’s  display  shown  on  the  PC  screen.  Animated 
graphics  permit  the  use  of  arrows  and  highlights  to  direct  the  learner’s  attention  to 
particular  switches  and  display  elements.  But  the  learner  can  enter  responses  EITHER 
through  the  touch  screen  on  the  PC  display,  or  on  the  micro-trainer  connected  to  the 
PC.  Finally,  the  learner  is  asked  to  perform  a  complete  task  using  the  micro-trainer. 

The  host  computer  can  monitor  the  student’s  work  and  determine  if  the  instructional 
task  has  been  mastered. 

The  combination  of  multimedia  PC  and  the  micro-trainer  gives  the  training  designer 
complete  freedom  to  select  the  most  appropriate  presentation  and  response  modes  for 
the  instruction.  The  hardware  can  support  any  training  strate^  from  rigid,  small  frame 
lock-step  programming  to  free  play  activity  bordering  on  full  simulation  of  the  device. 
This  advance  is  made  possible  by  the  use  of  simple,  standard  interfaces  to  the  micro¬ 
trainer,  its  small  size  and  portability,  and  the  availability  of  a  rich,  productive 
programming  environment. 

Support  and  Dealing  with  Change.  A  common  argument  put  forward  in  support  of 
using  actual  tactical  equipment  in  simulators  and  trainers,  instead  of  using  replicas  or 
emulators  as  is  recommended  here,  is  that  support  of  the  micro-trainer  is  difficult.  In 
particular,  a  major  concern  is  tracking  a  change  in  the  operational  system.  In  a 
simulator  built  up  from  actual  aircraft  equipment,  operational  system  field  upgrades  are 
tracked  by  making  a  change  to  the  corresponding  component  in  the  aircraft.  A  good 
example  is  the  updating  of  navigation  data  bases  on  aircraft.  The  data  bases  are 
updated  in  the  simulator  just  as  they  are  in  the  actual  aircraft.  Typically  the  simulator  is 
assigned  a  tail  number,  so  that  the  simulator  gets  any  change  that  the  particular  aircraft 
sees. 

It  is  true  that  the  ability  to  track  system  modifications  is  extremely  important.  Since  the 
life  of  a  trainer  may  approach  the  life  of  the  aircraft— 10  to  15  years  or  more-this  cost  of 
changes  can  be  higher  than  initial  acquisition  cost.  The  experience  with  micro-trainers 
so  far,  however,  is  that  support  for  upgrade  and  maintenance  in  general  is  significantly 
helped.  It  is  certainly  greatly  improved  over  earlier  trainers  where  every  change 
involved  custom  code  and  rewiring. 

On  new  avionics  displays,  field  upgrades  typically  involve  modifications  to  the  software 
in  the  avionics.  A  new  display  item,  a  change  in  color  or  appearance-these  changes  are 
typically  very  easy  to  implement  in  a  micro-trainer,  if  the  trainer  software  was  done 
using  a  language  like  C,  and  if  the  development  environment  for  the  trainer  was 
acquired  along  with  the  trainer.  In  other  words,  it  is  easy  to  make  modifications  if  the 
change  does  not  involve  hardware  change,  and  if  the  user  owns  the  tools. 

Large  hardware  upgrades  involve  extensive  resources,  but  are  typically  less  expensive  to 
implement  on  the  trainer  than  on  the  operational  system.  One  issue  planners  need  to 
be  aware  of  is  the  lead  time  between  the  availability  of  the  operational  change 
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information,  and  the  time  the  change  can  be  made  in  the  trainer.  Coordination  is 
required. 

Perhaps  the  most  difficult  kind  of  modification  is  a  small  hardware  change-for  example, 
the  change  of  a  switch  from  a  push  button  to  a  cover-protected  toggle  switch  on  an 
overhead  panel.  These  mods  may  be  more  expensive  on  a  micro-trainer  than  on  an 
operational  system.  But  overall,  because  the  number  of  major  system  upgrades  is 
comparatively  small  and  the  total  cost  savings  in  the  micro-trainer  are  large,  the  micro- 
trainer  will  still  show  significant  cost  advantages  over  tactical  equipment  based  trainers. 
Support  issues,  including  tracking  mods  to  the  actual  system,  can  be  fully  addressed  with 
well-designed  emulators  operated  in  solid  ILS  programs. 


ANALYSIS  OF  THE  PRINCIPAL  BENEFITS 


The  discussion  so  far  has  answered  the  "what  is  it"  question.  It  has  also  included  some 
discussion  of  how  the  micro-trainer  is  developed  and  used.  From  that  discussion  it  is 
possible  to  get  hints  at  its  principal  benefits  to  the  training  organization.  Now  it  is 
useful  to  explore  those  benefits  in  detail. 


CASE  EXAMPLES;  FOUR  WORKING  TRAINERS 

Each  of  the  trainers  discussed  below  has  been  developed  by  Regency  Systems  and 
delivered  to  a  customer.  They  vary  widely  in  type  of  device,  in  the  way  the  micro- 
trainer  is  implemented,  and  in  how  the  unit  is  used  in  training.  In  the  examples  it  is 
possible  to  see  both  the  common  elements  that  make  all  of  these  units  micro-trainers 
as  well  as  examples  of  the  variety  of  application  possible. 


Multi-function  Control/Display  Unit;  Cockpit  CDU. 

Background.  In  the  "glass  cockpit"  the  Multi-function  Control/Display  Unit 
(MCDU  or  CDU)  is  the  equivalent  of  the  computer  terminal  that  the  crew  inember  uses 
to  communicate  with  the  flight  computers  in  the  aircraft.  It  is  used  for  entering  flight 
data,  monitoring  the  flight  computers,  and  in  emergency  procedures.  It  may  also  be 
used  for  maintenance  functions.  Its  introduction  into  the  cockpit  caused  major  training 
problems  for  many  commercial  airlines.  To  some  extent  one  could  actually  argue  that 
this  training  need  created  the  market  for  the  modern  micro-trainer . 


Figure  2.  Regency  Systems'  Multifunction  Control  Display  Unit 
for  Contemporary  Commercial  Transport  Aircraft 
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All  of  the  key  features  discussed  above  are  present  ir. 

ne  MCDU 

he  MCDU  trainer  that  Rege  cy  p  diagnostics 

:  l;-='^-rbS^eTbrdanddis,ayof^ 

u  cJLnunicates  with  a  host  proce^^^^^^  ®”''.‘™"hke"the  one  used  on 

•  ;;  'IrbTJXred  wHh  an  AR1NC429  interface  just 

•  ritlilcUecuipment.  designed  the 

Por  a  ntiiitary  aitcrafb  ^  “ 

micro-trainer  to 

•bit  procedures  trainer.  taportance  of  a  high  degree 

-Srv?d^raSf“-^of.^^^  com— ions  and 

of  realism  and  benefits  aerivcu 

control  interface.  cutset  was  to  produce  an 

ReaU^  -nte  goal  of  f— f  ^  ™  „ary  controUnmrf^e  to  the  ftrst 

extremely  high  degree  procedures.  In  *^or  training  transfer, 

officer  in  Navigatto^  »d  Non  Nortn^^^^,,  an  pom 

fool  not  just  the  students,  but  , 

^  achieve  realism,  R^gen— — rinSing  the  <raio«-^;S;5‘^.S  -e 

f ;s'srr Scs.  r.S •' 

maker  of  the  prototype  device. 

prototype.  HUnlav  Not  only  is  the  real 

The  biggest  challenge,howevenw.—og^^- 

unit  a  displays.  After  considerable  semchi  g,^^|.^^^^  conmercial 

coating  typical  of  cocl^it  v  electromcs  So  by  us  g 

match  the  CRT  tube  itse  ,  remarkably  helpful  ^  ^lut  that  operates 

parts,  and  with  — p  ^  device  that  u^s  the  r^ght  ^ 

technical  afaff  "or^  6  mgj  nionitot.  In  addt  ,  S  identical  to  the 
“iroTsXme  t-ar  8— /otS  crsf—gn  definitely  traded  away  some 
amtearance  of  the  actual  display,  bo 
-cost  advantage  for  an  extremely  high 

outboard  peripheral  m  a  LB  l  e 
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.nrknii  nrocedures  trainer  Beyonri  those  two  applications,  Regency  recopiaed  that 

to  support  the  actual  atraaft  ^NC-429  The  requirement  that  the 

commercial  world  what  MIL-biU-npJ  ‘V  ^  ^  fit  the  form-factor  of  the 

unit  work  in  a  larger  trainer  meant  that  the  devtce  needed  to  fit  the  form 

prototype  MCDU. 

To  meet  all  these  needs.  Regency  decided  to  buM  ‘he  Hainer 
purpose  computer  (Intel  80286),  rather  than  a 

an  internal  bus,  a  full  operating  f  “"S  '^'chanL  mlt  for  various 

communications  interface  is  plugged  mto  the  b  ,  ^nmnuter  in  itself  If  one 

aoolications  The  point  is  that  the  trainer  is  a  complete  computer  in  . 

^mputer"  continue  to  gain  popularity,  even  more  flextbiltty  wtll  be  posstble  tn  micro 

trainers  that  are,  in  fact,  completely  self-contained  computers. 

Generalhable  Conclusions.  Tbe  idea  expressed  here  that  realUm  “td  co« 

dl^tor^pEoSria^f^^^^^^^^ 

Building  a  unit  that  fits  in  the  form  factor  of  the  actual  device  costs  mo  y. 

The  idea  that  the  miniaturization  of  personal  SSuv^C’s- 

imorovement  in  the  price  performance  of  commercial  computers-especially  s 

Z^yTuSests  tha”  micro-trainers  will  become  increastngly  attractive  in  the  future. 

A  Portable  Aircrew  Trainer 

of  the  required  training  and  practice  is  expensive. 

The  Portable  Aircrew  Trainer  (PAT).  To  provide  training  ^ 

cost  environment,  a  Maryland  firm,  CRT  Corporation  goal  w^^  to  support 

year  period  to  develop  a  “  ^  o/aTelf- 

all  phases  “bfrclx"'  ou^toS  nbcrXn^  and  display  panel,  and  a 

contained  PC  for  CBT,  an  otithoaro  mcro  i  j  ^  Corporation  on 

complete  courseware  environment.  Regency  bystems  worKeu  wun  k 

the  development  and  manufacturing  of  the  outboard  micro-traine  . 
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drawing  of  Micro-Trainer  Portion  of  CRT  Corp's  Portable  Aircrew  Trainer 
ee  large  rectangular  switch  banks  are  from  tactical  equipment.) 


The  main  purpose  of  the  outboard  switch  and 
positional/procedural  training  with  feedback, 
training  environment. 


display  panel  is  to  give  accurate 
It  is  just  one  part  of  a  fully  integrated 


Ponability.  A  major  goal  of  the  PAT  program  was 
the  needs  of  the  P3  squadrons,  it  was  essential  that  the  unit  be  small  and  self  container 

k  proTote  sWem  bSilt  several  years  ago  achieved 

rC  designe^^^^  portability  requires  mcreased  m^edness. 

If  the  unit  is  to  be  moved,  the  connectors,  internal  wiring,  and  the  mounting  ot  all 
compornti  need  extra  suength.  Even  things  like  the  paint  need  to  be  reconsidered  in 

building  a  portable  trainer. 

Use  of  a  Standard  Micro-Controller.  Key  to  Regency’s  strategy 
portable  trainer  low  cost  was  to  use  a  standardized  micro-controller.  Early  in^its 
Kopment  program.  Regency  realized  that 

controller  electronics  for  each  project,  costs  and  lead  times  P  P  .  Her 

Regency  created  a  standard  unit,  internally  called  the  General  Pu^ose  SermlContro 
(GPSC).  This  device  consists  of  a  family  of  COTS  cards  and  ^ackp  a  p  , 
Lme  developed  and  built  at  Regency,  that  can  be  configured  to  meet  a  broad  variety 

needs. 

In  the  PAT  micro-trainer,  a  three  board  GPSC  unit  provides  the  serial  interface  to  the 
PATS  CBT  host,  and  provides  programmable  control  of  the 

number  of  switches  and  indicators.  The  modular  character  of  the  GPSC  make  the 
internal  layout  of  the  micro-trainer  much  cleaner,  smaller,  and  more  reliable  than  we 
accomplLed  in  the  prototype.  The  modularity  of  design  has  “1^?  f 

software  and  the  switch  wiring  as  well,  so  that  maintenance  and  CBT  programming 

greatly  simplified. 

Generalized  Observations.  Regency  learned  a  valuable  lesson 
trainers  from  our  PAT  experience-lessons  that  we  have  generalized  to 
trainers  Modular  electronics  and  modularity  of  design  in  general  are 
for  reducing  costs  and  increasing  reliability.  Many  of  these  modules  meet  requirements 

for  COTS  acquisitions. 


AN/UYQ-21  (SQQ-89)  Trainer 

Background.  With  the  development  of  the  Aegis  and  the  SQQ-89  sonar  weapon 
systems,  tracing  planners  in  several  Navy  centers  began  to  realize  that  ®^tinued 
dependence  on  tactical  equipment  for  training  was 

tactical  units  fa  real  AN/UYQ-21  console  costs  over  $400,000),  plus  the  cost  ot  ormng 
to  r™  with  computers  from  other  parts  of  the  weapon  system,  got  People 

thinking  about  emulation  as  an  alternative  In  ‘“^Atton  the  mH  deployment  of 
systems  meant  that  large  numbers  of  people  would  need  to  be  trained  m  a  short  pe 
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RE^ncy  Systems’  AN/UYQ-21  (OJ-535) 

REgency  y  display;  Lower  displa>  is  no.o. 

ideo  or  tactical  graphics  or  CBT . 
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of  time.  So  the  number  of  systems  needed  for  training  became  large.  Several  hundred 
training  consoles  are  needed  for  SQQ-89  training  alone. 

As  a  part  of  a  modular  family  of  console  displays,  the  OJ-452  and  OJ-535  consoles  used 
in  the  AN\UYQ-21  were  natural  candidates  for  emulation.  Of  particular  appeal  to  one 
group  of  training  designers  was  the  idea  that  the  OJ-535  console  itself,  with  its  large 
display  monitors  and  trackball,  could  be  a  CBT  training  device  itself.  The  same  training 
device  would  be  used  to  emulate  the  functions  of  the  tactical  equipment  and  to  provide 
multimedia  CBT  instruction  as  well.  Merging  the  micro-trainer  and  multimedia 
functions  in  a  single  unit  is  an  innovative  training  concept. 


The  AN/UYQ-21  (OJ-535)  Emulator.  In  1989  Regency  was  awarded  a  contract  to 
develop  12  prototype  Interactive  Videodisc  Delivery  Systems  (IVDS).  These 
workstations  were  complete  multimedia  systems  with  level  III  interactive  videodisc, 
digital  audio,  high  resolution  color  graphics,  and  a  powerfol  funeral  purpose  PC.  ^fot 
the  components  were  housed  in  a  console  replica  of  the  OJ-535  with  the  /  O 
panels  installed.  These  panels  were  fully  active,  so  that  the  trainer  could  be  used  to 
emulate  the  functions  of  the  AN/UYQ-21  as  it  is  used  in  the  SQQ-89  Sonar  weapon 
system. 


In  1991,  the  Naval  Underwater  Systems  Center  awarded  a  contract  for 
version  of  the  IVDS  workstation  components,  also  to  Regency.  This  year  (1992)  NOS 
Indian  Head  is  awarding  a  separate  contract  for  the  OJ-535  console  replic^.  When  the 
workstation  components  are  integrated  into  the  console  replicas,  the  resulting  trainer  is 
a  powerful  emulator  as  well  as  a  full  featured  multimedia  training  system. 

The  principal  motivation  in  turning  to  micro-trainer  emulation  for  the  OJ-535  was  cost 
reduction.  In  the  SQQ-89  program  alone,  cost  savings  in  training  from  the  use  of 
emulation  will  exceed  one  hundred  million  dollars  in  acquisition  costs  over  ju^  three 
years.  Savings  over  the  program’s  15  year  life  will  be  more  than  double  that  figure. 

Note:  Additional  information  about  the  IVDS  project  can  be  found  in  two  papers  W  be 
presented  in  February  at  the  SALT  Interactive  Instruction  Delivery  conference  m  Mando 
"Integrating  Interactive  Tutorials  with  High-Quality  Equipment  Simulations  by  William  J. 
Yalen  of  Analysis  and  Technology,  Inc  (North  Stonington,  CT),  and  'The  Interactive 
Videodisc  Delivery  System:  Advanced  Training  Technology  for  an  Important  Nap/  Program 
by  Richard  Davis  of  Regency  Systems,  Inc  (Champaign,  IL)  are  both  on  the  Orlando 

program. 


Multimedia  Integration.  It  is  the  multimedia  integration  of  the  IVDS  that  makes 
it  unique  as  an  emulator. 


The  multimedia  characteristics  of  the  IVDS  workstation  include  almost  all  of  the 
features  of  a  contemporary  multimedia  PC  environment.  Among  these  are: 


Powerful  CPU  (33  MHZ  Intel  80486) 

High  resolution  (1280  x  1024)  graphics  display 
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Videodisc  pteyer  al  1280  x  lOi-l 

and  X-Wlndo«s  capab-Uiy 

_ _  The  system  do< 
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Mn-11  Cockpit  Autoflig 


397 


CAPT:  Cockpit  Autoflight  Procedures  Trainer 

f  rnckoit"  brought  about  a  revolution  in 

Background.  The  in  crew  load,  and  the  increased  need  for 

training  as  well  as  in  .  trafning  far  in  excess  of  what  can  be 

training  time  has  created  demand  for  tram  g  off-load  the  simulators, 

accommodated  during  aircraft  use  of  procedures  trainers. 

Fo" 

device  short  of  the  simulator. 

An  important  development  in 

the  FAA  of  a  new  classification  sy  ^  "Level  6"  devices,  meaning  they  have 

tier  system,  full  fli^t  simulator  motion  flight  controls  with  feedback,  and 

sophisticated  visual  systems,  mu  P  .  of  \he  aircraft  instruments  in  re^  time.  A 

Sr?"  no  mcion,  and  less  stringent  reqnirctnents 

for  controls  and  instruments. 

Th.  TAFT  is  a  Uvel  4  device.  It  has  no  visual  system  and  no  %ht  controls,  but  for 
S'de^  !t  — .  all  functions  of  the  aircraft  are  represented. 

The  MD-Il  Autoflight  f^eso'”  H’®  autoflight 

trainer  for  the  McDo^eU  Dougte  mmra  t 

features  of  the  MD-U's  6^“*  ,  “^°“^ode  control  panel),  and  the  major  flight 

trainers  for  the  Lv^disolav  and  engine  instruments/crew  advisoiy 

instruments  (prim^  flight  ^  ^iners  the  CAPT  uses  a  main  computer 

system).  In  addition  to  the  fUcht  control  computers  and  flight  management 

cockpit  procedures  trainer  based  on  actual 

airaaft  instruments. 

FuU  Integration  of  Simulator  ^“ChOT  “mg  Mu/tg^icro  rrajj'^  simuLtor 

i"e^Se1:T=larn"rcr  0^  o?re  ^nttre  trainer. 

This  technique  works  bemuse  of  So  it 

the  host  processor  end  of  ^PP^^n  ^  independently  of  the  other  parts  of  the  system, 
can  be  developed,  tested,  cL^  cSuniclte  with  other  devices  using  low 

In  the  training  system^,  each  umt  channels,  but  local  area  networks 

cost,  standard  connections,  me  •  ®  ^  ^5  as  vvgll  At  the  host  end,  the  job  of  creating 
and  aircraft  type  advance  of  compilers  and  software 

software  has  been  made  simpler  t  dy  Whereas  a  conventional 

development  tools-especially  for  C  and  C+  +  langu  g 
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trainer  may  represent  a  million  lines  of  FORTRAN  or  ADA  code  a  based 

machine  may  require  only  10%  as  much  code  written  in  C  or  C  +  +.  Furthermore,  the 
code  is  highly  modular  and  re-usable-an  important  consideration  for  maintenance  and 
long  term  supportability. 

Emulation  oj  Complex  Systems.  The  major  lesson  of  the  that  complex 

trainers  can  be  built  out  of  collections  of  simple  micro-tramers.  The  CA^  itself  can  be 

looked  at  as  a  collection  of  15  micro-trainers,  plus  a  control  computer.  The  open 
architecture  of  the  trainer  permits  addition  of  new  systems  with  minimum  change  What 

emulation  of  complex  systems  requires,  however,  is  a  ®  number 

together  vnth  a  well-designed,  well-managed  software  development  effort.  The  number 
of  functions,  and  the  complexity  of  interactions  among  various  parts  of  an  aircraft  or 
major  weapon  system  make  them  difficult  projects  for  emulation.  To  realize  the  cos 
reduction,  portability,  and  training  benefits  of  micro-trainers,  projects  need  ^ 
thought  out.  Of  particular  importance  is  the  generation  of  a  requirements  docume 
that  explicitly  states  what  functions  are  emulated,  and  how  the  actual  system  behaves. 

Generalized  Observation.  The  two  key  observations  to  be  made  from  the  C/^ 
are  that  large  systems  work,  even  when  they  are  complex,  and  that  careful  delineation  o 
the  functional  requirements  of  the  trainer  are  essential  in  large,  complex  emulations. 


The  Expanding  Role  of  Micro-Trainers.  The  key  points  set  out  in  the  abstract  have 
been  demonstrated.  Micro-trainers  provide  effective  emulation  in  a  broad  r^ge  o 
training  applications-even  complex  systems  such  as  cockpit  simulators.  Significant  cost 
reductions  are  achievable.  Continuing  advances  in  rnicro-electromc  and  training 
technology  promise  further  cost  reduction  and  technical  advances  in  micro-tramers. 
Increased  budget  pressures  in  training  suggest  that  trainers  will  seek  out  the  advantages 
of  low  cost,  portability,  and  flexibility  that  micro-trainers  provide. 

Notes  for  Contracting  and  Procurement  The  authors  have  revieived  over  twenty 
solicitations-principally  from  the  Navy  and  Air  Force-for  airborne  and  chipboard 
trainers  that  could  involve  micro-trainers.  Based  on  experience  in  those  solicitations, 
and  in  the  author’s  work  on  the  projects  used  as  examples  above,  the 
recommendations  are  offered  to  those  writing  solicitations  that  include  micro-tramers. 

f  -  ■  ' 


for  a  requirements  document  to  be  generated  by  the  vendor._  It  is  essential  that 
there  be  absolute  clarity  about  what  functions  the  emulator  is  to  emulate. 

Without  such  clarity  conflicts  are  almost  certain  to  arise  a^ut  what  the  unit  is 
supposed  to  do.  It  is  reasonable  for  10%  of  the  project  NRE  to  go  mto  the 
requirements  document;  it  will  save  money  in  the  long  run  It  is  not  sufficient,  by 
the  way,  to  simply  state  that  the  training  device  will  have  the  same  functionality 

as  the  tactical  equipment. 
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.  solicitation  should  include  a  CUN  forjhe 

of  the  trainer  software  SolicitatiOTS  y  q  g^j 

documented  code  and  even, j  purchased,  the  trainer’s  code 
the  development  environment  o  a  so  pu 

r^onan"  ^hTe  »  acquire  the  toois  used  to  create 

the  code. 

Acquisition  of  code*1)rpro«'ded.  “in  ‘the  first  cJt.  the 

commeraal  software.  But  negative  results  may  occur.  Some 

development  cost  may  be  very  g  .  ujhq  because  they  refuse  to  give  up 

qualified  vendors  will  not  respond  to  such  ^  ^  ^  ^  competition 

Sieir  rights.  Or,  it  “  SS  “  "rfbe  an  option,  and 

Xb““touirbe  considered  separately  in  cost  evaluations. 

.  Preconceived  notions  about  *"  Xn j^To^rapid^” 

raise  costs  and  reduce  effectiveness  i  almost  certainly  drive  up 

that  any  unnecessary  constrmnts  put  lu  efforts.  As  an  example, 

costs  aid  limit  the  abiUty  of  the  micro-trainer.  Some 

a  recent  solicitation  required  fwr  processws^m  performance  to  be 

bidders  had  access  to  technology  P  P  ^  requirement  was 

achieved  with  only  one  less  effective 

to  raise  the  bids  by  over  5%  ^d  processors 

fo  w  J  ZTL  the  bidder  to  describe  the  micro-trainer 
architecture  in  detail  in  their  techmcal  response. 

.  Clean,  establish  requirements  for  the 

gets  at  two  '^^^“P^Stetactical  system;  the  people 

solicitation  are  typically  very  .  ^rly  so  familiar.  This  means 

srti:f^«m"“»g 01  iT" 

of  expectations  in  planning  the  device. 

The  other  problem  is  that  cost  and  the  °he  ven?oMs  strongly  motivated  to 

tUnicallyVptable/lowestcostcomp^^^^^^^^^^ 

sacrifice  fidelity  to  save  mon  y.  ^  possible  about  what  they  want, 

authors  of  the  solicitation  nee  specified  as  requirements  are;  finish, 

Src^XnS?acfion  (stro'Le  length,  pressure,  elicit),  lamp  color, 
lamp  brightness,  and  dimensions. 
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o  Whenever  possible,  the  contracting  officer  should  make  ^arrangements  for 
bidders  to  get  access  to  an  actual  unit  and  to  full  documentation  of  a  tactical 
systemio  It  is  almost  universal  experience  among  training  device  vendors  that 
seeing  the  unit  is  even  more  valuable  than  the  documentation.  Seeing  the 
prototype  reduces  risk  for  the  training  vendor.  Clarification  about  how  a 
particular  switch  works,  or  how  a  particular  function  operates  can  help  the  vendor 
identify  the  most  cost  effective  way  to  build  the  micro-trainer.  This  reduced  risk 
will  translate  directly  into  lower  acquisition  cost.  The  same  is  true  post  award. 
The  government  should  furnish  access  to  the  prototype  (within  limits)  as  part  of 
the  information  available  to  the  successful  bidder.  Documentation  of  the 
prototype  is  essential  in  making  sure  that  individual  functions  are  correctly 
supported. 

«  ADA  should  be  avoided  in  favor  of  C  or  C  +  + .  One  key  to  the  low  cost  of 
micro-trainers  is  the  availability  of  commercial  development  tools  for  micro¬ 
controllers.  The  tools  are  part  of  the  project  cost.  Since  the  corresponding  tools 
in  ADA  are  much  more  expensive,  and  not  nearly  as  productive  as  the  highly 
developed  C  tools,  ADA  is  a  weak  choice  for  micro-trainers  compared  to  C  or 
C++.  Since  these  micro-trainers  are  NOT  tactical  equipment,  the  argument  for 
waiving  ADA  is  strong. 

»  Keep  the  mkro-traimer  am  the  training  device  category.  Procurernent  rules 
and  organizational  pressures  often  put  the  contracting  officer  in  the  position  of 
wanting  to  purchase  the  device  as  data  processing  equipment  or  as  audio-visual 
equipment.  The  danger  here  is  that  firms  with  no  training  experience  will 
respond.  Since  they  have  no  experience  in  building  training  devices,  the  customer 
will  inherit  the  problems  of  the  vendors  first  effort  at  a  micro-trainer.  Vendor 
qualification  is  important,  both  in  terms  of  assurances  of  product  quality,  as  well 
as  in  helping  the  vendor  work  effectively  with  the  customer  to  meet  the  real 
training  need. 

«>  Allow  adequate  time  between  "First  Article"  delivery  and  the  first 
production  shipmerat  to  accommodate  changes.  There  are  almost  certain  to  be 
revisions  to  the  software,  and  there  may  be  revisions  to  the  hardware  of  the 
trainer  following  evaluation  of  the  First  Article  shipment.  It  is  important  that 
there  be  enough  time  in  the  schedule  to  permit  those  changes  to  be  integrated 
into  the  production  systems.  Allowing  only  30  days  between  First  /yticle  and 
production  is  probably  not  adequate  time  to  permit  adequate  technical  review 
AND  to  put  changes  in  place. 

•  Real  Time  operating  systems  are  generally  not  a  significant  advantage.  In 
the  experience  of  the  authors,  "real  time"  operating  systems  in  a  trainer  greaUy 
increase  its  cost,  but  produce  little  or  no  observable  benefit  in  the  training.  One 
reason  for  this  is  that  computers  today  are  so  fast  and  so  inexpensive  that  it  is 
almost  always  easier  and  cheaper  to  add  computer  speed  than  it  is  to  add  real 
time  operating  system  control. 


.  Keep  the  vendor  involved  in  the  ILS  program.  The  vendor  is  in  the  best 
position  to  assist  the  user  with  maintenance,  upgrades,  and  even  training 
development.  While  it  is  common  to  purchase  the  micro-trainer  under  one 
vehicle,  and  training  development  under  another,  it  is  important  that  the  vendor 
be  able  to  provide  specialized  services  after  the  micro-trainers  are  delivered. 
Viewing  the  micro-trainer  acquisition  as  a  single  equipment  acquisition  ignores 
important  program  needs  for  integration  and  support. 
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seaioD  from  the  training  systems  development 
industry  and  included  the  following  companies, 
o  CAE-Link  Corp 
o  ECC  International  Corp 
o  FlightSafety  Services  Corp 
o  Hughes  Simulation  Systems  Inc 
o  Loral  Defense  Systems  Division 
o  McDonnell  Douglas  Trtuning  Systems  Inc 

Membership  from  the  SPO  consisted  of  two  func¬ 
tional  experts  representing  the  disciplines  of 
Engineering  and  Test  Management. 

The  Cumberland  Group,  a  subsidiary  of  Armco 
Steel,  conducted  an  intensive  four  day  workshop  to 
train  CPT  members  to  analyze  and  improve  the 
process.  The  Training  Systems  SPO  agreed  to  fund 
the  training  for  each  CPT  member.  Team  mem¬ 
bers  gained  a  common  understanding  of  the  CPT 
purpose  and  were  able  to  come  to  a  consensus  on 
how  the  acceptance  test  process  is  a  summation  of 
activities  which  must  be  completed  in  the  course  of 
providing  a  product  or  service.  Effective  working 
relationships  were  established  and  the  team  struc¬ 
ture  was  created.  Training  provided  the  beginnings 
of  an  understanding  of  the  Cumberland  Process 
Improvement  Model  methodology. 

THE  PROCESS  IMPROVEMENT  MODEL.  The 
objective  of  process  management  is  to  focus  on  the 
customers,  determine  what  their  requirements  are, 
analyze  how  work  will  be  accomplished,  and  identi¬ 
fy  and  eliminate  the  sources  of  waste  in  the  pro¬ 
cess. 

The  Cumberland  Process  Improvement  M^el 
consists  of  five  primary  steps  leading  to  the  elimi¬ 
nation  of  nonvalue  added  components.  The  model 
stresses  that  quality  problems  are  very  often  rooted 
in  the  process  that  produced  them.  A  change  in  the 
process,  therefore,  is  required  to  achieve  meaning¬ 
ful  improvement  and  not  just  merely  eliminate  the 
symptoms.  This  is  the  foundation  upon  which  the 
process  management  approach  to  quality  improve¬ 
ment  is  built.  Following  is  a  brief  description  of 
each  step  in  the  process  improvement  model. 

Dfffinition  of  the  Improvement  Opporluni^:  To 
develop  a  clear  understanding  of  the  team’s  task, 
desired  expectations  were  clarified.  A  flow  chart  of 
all  acceptance  test  activities  was  constructed  to 
better  understand  the  proce^.  Finally,  indicators 
(measures)  of  improvement  were  agreed  upon  to 
guide  the  team  as  it  searched  for  areas  of  process 
adjustments. 


r>ai^  rollection:  A  questionnaire,  based  on  the 
measures  of  improvement  was  developed  and  used 
for  data  collection  to  move  from  the  statement  of 
the  problem  to  a  more  complete  description  of  the 
current  process.  Benchmarking  of  similar  pro¬ 
cesses  was  initiated.  Performance  measures  of  sev¬ 
eral  completed  Government  test  programs  were 
then  documented  for  subsequent  process  analysis. 


Analysis  of 


s:  Data  col- 


lected  from  the  previous  step  was  used  to  identity 
and  prioritize  waste  and  focus  efforts  on  the  high 
payback  areas.  Waste  is  defined  as  any  activity  that 
does  not  add  value  to  the  process  and  was  riewed 
as  the  primary  opportunity  for  improving  the  pro¬ 
cess.  In  the  final  element  of  this  step,  the  root 
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n^v>-lnnment  of  Solutions:  The  intent  here  was  to 
generate  alternative  ways  to  eliminate  root  caus^ 
of  waste.  The  team  concentrated  on  ways  to  signif¬ 
icantly  change  the  process  instead  of  merely  mak¬ 
ing  minor  adjustments.  With  no  assumed  con¬ 
straints,  the  "perfect"  process  was  visuali^d  to 
form  an  understanding  of  what  could  really  be 
achieved,  even  if  in  stages  rather  than  aU  at  once. 

Improvement  Recommendations:  This  final  step 
was  designed  to  improve  the  current  accept^cc 
test  process  through  a  series  of  recommendations 
resulting  from  solutions  developed  di^g  prior 
analysis  steps.  Continual  improvement  is  made  by 
planning  the  modification,  engaging  the  plan,  then 
checking  and  making  adjustments  based  on  the  re¬ 
sults. 

definition  of  THE  IMPROVEMENT 
OPPORTUNITY 

The  process  of  developing  a  clear  understanding  of 
the  CPTs  task  and  clarifying  expectations  for  im¬ 
provements  in  simulator  testing  was  the  team  s  fust 
major  assignment.  The  mission  statement  that 
follows  was  developed  in  order  to  clearly  define  the 
purpose  and  reason  for  existence  of  the  CPT. 

"We  are  the  YW/Industry  Partnership  CPT, 
committed  to  continuously  identify  and  promote 
implementable  approaches  to  minimize  the  cost 
and  time  required  for  acceptance  testing  of  aircrew 
and  maintenance  training  devices  and  to  insure  that 
these  devices  support  the  users’  needs. 

The  test  process  was  initially  defined  as  the  pri¬ 
mary  means  by  which  a  training  device  is  evaluated 


404 


for  compliance  of  ihe  design/product  apmst  re¬ 
quired  characteristics  and  system  performance. 
Through  verification,  validation,  and  authentica¬ 
tion,  the  adequacy  of  performance  characteristic 
are  determined  along  with  identification  of  defi¬ 
ciencies  in  system  performance.  Acceptance  test¬ 
ing  is  defined  as  any  and  all  contractor  and 
Government  activities  performed  to  verify  device 
conformance  to  specified  system  subsystem  per¬ 
formance  requirements. 

The  test  process  provides  contract  closure,  and  al¬ 
lows  training  initialization.  Yet,  despite  its  impor¬ 
tance,  the  test  process  and  accompanying  test  doc¬ 
umentation  has  been  reported  as  byzantme  at  best. 
Many  myths  and  misinformation  abound.  There  is 
widespread  belief,  for  example,  in  the  following; 
o  Acceptance  testing  contributes  to  schedule 

delay  „ 

o  The  Government  must  witness  all  accep¬ 
tance  tests 

The  flowchart  shown  in  figure  1  depicts  typical  test 
activities  and  may  vary  somewhat  depending  on  a 
particular  program’s  requirements.  Review  of  the 
existing  process  revealed  that  there  were  several 


test  repetitions,  multiple  Test  Readiness  Reviews 
(TRRs),  and  numerous  possible  delay  paths. 


Process  indicators  were  used  to  measure  the  per¬ 
formance  of  the  acceptance  test  activities.  Two  ap¬ 
proaches  were  used  to  generate  the  proce«  mdica- 
tors.  First,  the  CPT  membership  produced  a  list  of 
parameters  which  measured  the  performance  of 
The  acceptance  test  effort  based  on  specific  tes^g 
experiences.  The  second  approach  w^  to  define 
measures  relating  directly  to  the  delay  loops  m  the 
process  flowchart.  FmaUy,  the  two  lists  were  eval- 


a  If  the  test  process  is  revised,  will  the  pro¬ 
posed  indicators  be  able  to  measure  the  improve¬ 
ment? 

b.  Can  the  indicator  bcTncasured  in  real  terms 
using  objective  results? 

c  Can  data  be  obtained  from  the  companies, 
is  it  something  likely  to  be  measured  and  retamed 
as  part  of  the  existing  testing  process? 


figure  1.  CURRENT  SIMULATOR 
TEST  APPROACH 
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d.  What  is  the  most  important  data  to  request? 
The  fmal  list  of  process  indicators  totaled  38  dis¬ 
tinct  measures.  The  CPT  focused  on  the  four  top 
indicators  to  collect  data  for  further  analysis. 

Test  Milestones  Met  or  Delayed;  The  first  indica¬ 
tor  was  based  on  schedule  milestones  required  to 
conduct  a  test  program.  Program  and  contract 
schedule  events  were  chosen  in  anticipation  that 
such  data  would  be  recorded  and  available. 

Number  of  Test  Discrepancies.;  The  number  of 
Test  Discrepandes  (TDs)  generated  during  accep¬ 
tance  testing  is  a  measure  of  the  traimog  device 
quality.  To  get  more  insight  into  the  causes  of 
TDs,  data  was  requested  to  indude  total  number  of 
TDs,  number  of  TD  re-submits,  number  of  post 
seU-off  TDs,  and  number  of  TDs  out-of-scope. 


example,  only  limited  data  was  provided  for 
maintenance  trainer  programs  apparently  due  to 
less  stringent  test  requirements.  Comparison  data 
was  therefore  nonexistent.  As  a  result,  six  military 
programs  remained  for  further  evaluation. 

BENCHMARKING,  The  concept  of  "who  does  it 
best"  is  known  as  benchmarking.  The  approach 
was  to  identify  possible  candidates  for  the  CPT  to 
evaluate  as  "best."  The  benchmark  selection  crite¬ 
ria  included  stringent  testing  and  the  use  of  com¬ 
mercial  practices  relating  to  the  CPT  process  indi¬ 
cators.  Candidate  sources  were: 
o  Airline  simulator  programs 
o  Simulation  industry 
o  TQM  award  winners 
o  Other  TQM  intense  companies 
o  NASA  simulators 


Number  of  Davs  in  Test:  The  purpose  of  this  indi¬ 
cator  was  to  isolate  schedule  variances  by  measur¬ 
ing  duration  of  key  test  events  (planned  days  vs.  ac¬ 
tual  days).  From  the  results,  the  CPT  selected 
three  phases  to  measure  test  duration  as  a  perfor¬ 
mance  indicator.  These  were  in-plant  development 
tests  and  on-site  acceptance  and  operational  eval¬ 
uations. 

Test  Documentation:  The  CFT  membership  con¬ 
sidered  test  documentation  exce^ive.  The  size  of 
the  test  procedures,  i.e.,  number  of  pages,  was  the 
means  used  to  measure  this  excess.  In  addition,  the 
detail  to  which  the  procedures  were  written  was 
measured  by  the  number  of  test  steps  per  page. 

DATA  COLLECTION 

After  settling  on  which  indicators  to  be  used,  it  was 
then  necessary  to  consider  the  possible  sources  of 
data  for  the  information  the  CPT  needed.  In  de¬ 
termining  the  selected  programs,  the  CPT  focused 
on  recently  completed  test  programs  and  the  likeli¬ 
hood  of  gathering  accurate  data.  Finally  a  ques¬ 
tionnaire  whidi  focused  specifically  on  the  process 
indicators  was  developed  and  used  to  gather  sup- 


The  raw  data  from  the  questionnaire  was  analyzed 
and  incorporated  into  summary  sheets.  Several 
very  important  adjustments  were  made  to  produce 
the  summary  sheets.  The  first  adjustment  was  to 
eliminate  companies/programs  that  did  not  re¬ 
spond  to  the  questionnaire  and  those  who  felt  their 
process  was  different.  In  addition,  programs  were 
excluded  if  sufficient  data  was  not  available.  For 


The  replies  to  the  CPT  membership  inquiries  and 
questionnaire  were  extremely  poor.  Successive  fol¬ 
low-ups  by  team  members  did  little  to  elicit  further 
responses.  Many  indicated  they  felt  their  testing 
approach  was  sufficiently  different  to  make  it  un¬ 
suitable  for  our  purposes.  It  rapidly  became  appar¬ 
ent  that  integration  and  test  of  a  full  flight  simula¬ 
tor  is  a  uniquely  challenging  task  not  commonly  en¬ 
countered  in  other  industries. 


At  that  point  the  team  decided  to  focus  on  the 
commercial  airline  simulator  industry  as  the  candi¬ 
date  for  "best".  This  was  based  on  the  fact  that  they 
buy /build  a  product  similar  to  the  Air  Force,  use 
rnmmercial  standards,  and  must  pass  stringent  ac¬ 


ceptance  testing  conducted  for/by  the  FAA.  Seven 
commercial  devices  provided  data  considered  ade¬ 
quate  for  benchmarking. 


ANALYSIS  OF  IMPROVEMENT 
OPPORTUNITIES 

There  is  a  fine  distinction  between  a  problem  and 
an  opportunity.  In  this  phase  of  the  CPT  effort,  as 
problems  were  substantiated,  opportunities  became 
apparent.  The  predominant  issue  was  then  to  fo¬ 
cus/select  opportunities  that  satisfied  the  mission 
statement. 


After  being  reviewed  for  omissions,  the  raw  data 
was  organized  for  the  purpose  of  identifying  waste 
areas  in  the  test  process  and  subsequently  deter¬ 
mining  the  root  causes.  The  data  was  grouped  ac¬ 
cording  to  the  four  process  indicators  and  studied 
for  information  and/or  conclusions  that  could  be 
drawn  from  the  data  sets.  The  data  was  plotted  to 


406 


obtain  a  visual  representation  and  studied  to  iden¬ 
tify  relationships,  trends  and  observations. 

Each  chart  was  individually  reviewed  to  search  for 
waste  areas  in  the  test  process.  Graphical  analysis 
assisted  in  producing  a  better  definition  of  the 
problems.  The  team  used  tabular  data,  histograms, 
the  process  flowchart,  and  comments  on  the  ques¬ 
tionnaires  for  identification  of  process  waste  areas. 

For  the  reader  with  greater  interest  in  the  experi¬ 
mental  design  the  complete  data  collection  and 
analysis  methods  are  contained  in  USAF  TR-90- 
5000,  12  Dec  90,  "Process  Improvements  in 
Training  Device  Acceptance  Testing  •  A  Study  in 
Total  Quality  Management’  available  from 
Defense  Technical  Information  Center  (DTIC), 
Cameron  Station,  Alexandria  VA  23661. 

Fight  critical  waste  areas  were  identified  for  subse¬ 
quent  root  cause  analysis.  Root  causes  were  un¬ 
covered  by  systematically  questioning  why  the 
waste  exists  until  the  root  cause  is  identified.  This 
was  done  by  asking  "why"  five  times.  This  tech¬ 
nique,  while  quite  basic,  allowed  root  causes  to  be 
determined  for  each  waste  area. 

DEVELOPMENT  OF  SOLUTIONS.  The  solu¬ 
tions  described  arc  based  on  eleven  months  of  in¬ 
tensive  study,  data -collection,  and  analysis.  With 
the  knowledge  obtained  up  to  this  point,  the  cur¬ 
rent  test  process  flowchart  was  revisited.  By  re¬ 
moving  all  constraints,  bias,  and  myths,  then  apply¬ 
ing  insight  gained  from  the  data,  an  idealized 
flowchart  was  generated  to  visualize  a  test  process 
void  of  identified  wastes.  This  new  flowchart,  in 
conjunction  with  the  information  gained  from 
identif^ng  root  causes,  provided  the  basis  for  de¬ 
veloping  solutions.  Although  the  solutions  are 
specific  in  nature,  they  are  not  intended  to  be  per¬ 
ceived  as  the  only  solution  but  rather  as  the  CPT’s 
suggestions  based  on  the  research  conducted.  It 
should  be  noted  that  all  solutions  had  a  consensus 
of  the  CPT  membership. 

Waste  Area  1:  Delay  in  start  of  tgSL 

The  following  root  causes  were  identified  by  the 

CPT  as  being  directly  related  to  test  delays: 

o  Late  Government  identification  of  mini¬ 
mum  training  needs 
o  Poorly  defined  requirements 
o  Incomplete  design 

-  Requirements  not  complete 

-  Data  not  available 

-  Resources  not  available 


-  Inefficient  implementation  of  new  tech¬ 
nology 

o  Manufacturing  not  complete 

-  Government  Furnished  Equipment 
(GFE)/Contractor  Furnished  Equip¬ 
ment  (CFE)  not  available 

-  Inadequate  subcontractor/vendor  man¬ 
agement 

o  Hardware/Software  Integration  jn  process 
measurement  criteria  lacking 

Program  delays  due  to  late  Government  identifi¬ 
cation  of  training  needs  often  cause  the  program 
planning  phase  of  the  procurement  process  to  be 
incomplete.  Inadequate  research  during  this  period 
results  in  poorly  defined  phenomenon  during  later 
stages  of  design  development.  The  problem  is 
further  compounded  when  the  contractor  accepts 
these  nebulous  requirements  and  consequently  fails 
to  perform  to  Government  expectations.  Thorough 
completion  of  the  training  requirements  analysis 
prior  to  the  release  of  the  RFP  will  greatly  assist  in 
well-defined,  realistic  requirements  to  provide  a 
sound  basis  for  contractor  scheduling,  pricing,  and 
technical  performance. 

Problems  associated  with  the  contractor’s  failure  to 
complete  the  design  prior  to  testing  due  to  incom¬ 
plete  data  may  be  decreased  or  eliminated  by  iden¬ 
tifying  mission  data  early  during  the  program  and 
establishing  joint  contractor /Government  interpre¬ 
tation.  This  interpretation  should  then  be  formally 
included  in  the  Design  Criteria  List  (DCL).  Early 
involvement  of  Sub  ject  Matter  Experts  (SMEs) 
will  also  help  alleviate  problems  associated  with  a 
lack  of  data  by  providing  an  "on-line"  data  source 
during  the  design  development  phase.  In  addition, 
implementation  of  the  data  generation  and  man¬ 
agement  principles  identified  in  the  Simulator  Data 
Integrity  Program  sponsored  by  the  Training 
System  SPO  under  contract  AF33657-88-C-2168 
should  be  considered  to  ensure  accurate  and  com¬ 
plete  data  is  provided  by  the  weapons  system  prime 
contractor. 

The  problem  of  unavailable  resources  centers 
around  delays  caused  by  events  leading  up  to  Md 
including  Hardware/Software  Integration  which 
have  been  determined  to  be  especially  significant 
by  this  CPT.  For  example; 

o  Hardware/Software  Integration  suffers  from 
poor  planning  and  implementation. 

o  Ability  to  manage  the  Hardware/Software 
Integration  process  has  been  lacking. 
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o  Start/stop  criteria  and  in-process  measure¬ 
ment  tools  have  been  non-existent. 

Schedule  risks  associated  with  utilizing  new  tech¬ 
nology  in  the  device  design  can  be  mitigated  by  de¬ 
veloping  prototype  testing  procedures  to  mature 
the  technology  prior  to  Hardware/Software 
Integration.  These  procedures  can  be  reduced  on 
subsequent  production  quantities  as  the  risk  of  the 
technology  decreases. 

GFE  availability  problems  can  be  reduced  by  im¬ 
plementing  a  system  in  which  the  Air  Force  pro¬ 
cures  the  required  training  system  components 
from  the  prime  weapons  system  contractor  as  soon 
as  GFE  requirements  are  known.  An  alternative  is 
to  have  the  Government  include  in  the  weapon 
systems  contract  the  requirement  to  enter  into  an 
associate  contractor  agreement  with  the  simulator 
contractor  to  supply  the  necessary  components. 
Alternatives  to  using  GFE  should  be  explored  such 
as  the  use  of  commercial  components  which  are  es- 
sentiaUy  equivalent  to  MIL-SPEC  hardware  items. 

Inadequate  subcontractor/vendor  management 
problems  are  not  in  the  customer’s  direct  line  of 
responsibility;  however,  the  Government  can  influ¬ 
ence  the  prime  contractor  to  address  this  area  to 
reduce  the  risk  to  testing.  Suggestions  by  the  CPT 
for  industry  improvements  include: 

o  Avoid  multi-level,  multi-party  subcon¬ 
tractor  arrangements  on  major  device  components 
where  the  actual  supplier  has  no  direct  link  to  the 
prime 

o  Establish  a  strong  subcontractor/  vendor 
management  team  to  take  responsibility  for  the 
supplier’s  performance 

o  Use  on-site  representatives  when  necessary 
to  closely  monitor  supplier  performance 
o  Use  Material  Requirements  Planning 
(MRP)  packages  to  help  schedule  vendor  delivery 
o  Develop  reliable  second  sources  for  high 
risk  components 

o  Insist  on  monitoring  and  reviewing  major 
subcontractor  performance  on  a  regularly  sched¬ 
uled  basis  to  identify  potential  problem  areas 

Waste  Area  2:  Redundant  testing. 

The  following  root  causes  were  identified  as  con¬ 
tributing  to  the  problem  of  redundant  testing: 
o  No  Government  recourse  after  buy-off 
o  Improper  engineering  test  procedures 

-  En^neering  procedures  not  repeatable 

-  Results  not  documented 


The  customer  typically  views  acceptance  testing  as 
his  "one  and  only  shot"  at  discovering  all  system 
problems.  This  results  in  aggressive  testing  by  the 
Government  to  ensure  the  continued  performance 
of  the  simulator  throughout  the  required  life  cycle. 
The  contractors  can  instill  confidence  in  their 
product  and  thus  lessen  the  need  for  extensive 
testing  by  providing  a  more  comprehensive  per¬ 
formance  warranty  package  similar  in  scope  to 
those  currently  available  to  commercial  airlines. 

Redundant  testing  often  occurs  as  a  result  of  poor 
contractor  testing  procedures.  The  Government 
does  not  accept  Contractor  Verification  Testing 
(CVT)  results  as  "final"  and  usually  reruns  a  sub¬ 
stantial  portion  of  contractor  test  procedure. 
Confidence  in  contractor  test  results  can  be  estab¬ 
lished  by  increasing  the  quality  of  in-plant  testing 
and  including  advisory  SME  involvement  during 
CVT.  This  solution  prompts  the  contractor  to  con¬ 
duct  in-plant  tests  which  are  repeatable  and  well- 
documented.  Consistent  contractor  test  results  will 
increase  the  likelihood  of  Government  acceptance 
of  the  data  generated  and  eliminate  the  need  for 
repeating  previous  contractor  testing.  Failure  to 
properly  perform  and  document  CVT  also  results 
in  jeopardized  on-site  device  performance  and  re¬ 
duced  profit  due  to  attending  schedule  delays  and 
additional  contractor  resources  necessary  to  up¬ 
grade  the  device  to  an  acceptable  condition. 

Wa.ste  Area  3:  Detailed  customer  subsv.sLeni 
Performance  verification. 

The  CPT  identified  the  following  root  causes  as 
contributing  to  overly  detailed  performance  verifi¬ 
cation  testing: 

o  Contractor  test  results  not  available  or 
documented 

o  Traditional,  bottoms-up  test  techniques 

o  Performance  risks  associated  with  new 
technology 

The  need  for  detailed  Government  performance 
verification  to  the  subsystem  level  can  be  elimina¬ 
ted  by  instituting  improved  contractor  test  proce¬ 
dures.  Prototype  tests  should  be  developed  for 
high  risk,  new  technologies  prior  to  Hardware/ 
Software  Integration  until  a  satisfactory  confidence 
level  is  reached.  Traditional  bottoms-up  testing 
should  no  longer  be  performed  by  the  Government. 
Instead,  these  procedures  should  be  completed 
during  contractor  testing.  The  procedures  and  test 
results  should  be  thoroughly  documented  for  Gov¬ 
ernment  review,  thus  allowing  one  time  cost  effec¬ 
tive  and  efficient  system  level  acceptance  testing. 
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problems  throu^  the  use  ^ 

agement  persormel  are  available 

possible,  and  ensure  The  determination 

in  each  specific  jjjto  Government  testing 

of  acceptable  risk  to  ^  niore 

with  known  problems  made  at 

SrpX%:Singtesttoavo-.^ 

ruptions,  and  waste. 

A  luck  of  ore/cre 

for  test  ‘ntc"nP‘‘°“^. ,  ing  requirements 

,hto  protolem  toy  ^  ,hu  wcuponu  sys- 

when  ordermg  components 
tem  manufacturer. 

.  •  c  thp  root  cause  of  many  test 
Poor  system  analysis  is  re¬ 
interruptions  as  unre  established  to  re¬ 

sult.  An  ad  ho-^^^u^nfiiSng  both  contractor 

solve  show  ^^°PP®  ^  required, 

and  Air  Force  resources  as  q 
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Modification  or  new  construction  of  a  facility  is 
most  often  accomplished  via  the  Military  Construc¬ 
tion  route  (33(K)  appropriation  funding).  However, 
the  ability  to  use  RDT&E  (36(K)  appropriation 
funding)  should  be  exploited  where  new  facility 
construction  is  a  requirement.  AFR  80-22  states 
that  RDTT&E  funds  may  be  used  to  acquire  indus¬ 
trial  and  RDT&E  facilities  needed  by  contractors 
to  fulfill  R&D  contraas  as  authorized  by  10  USC 
2353.  This  has  been  interpreted  to  mean  that 
where  a  facility  is  needed  by  a  contractor  in  order 
to  perform  tasks  required  by  a  R&D  contract,  that 
facility  may  be  provided  through  this  funding. 

7;  M.iltinle  test  readinS§§-radgw§.  . 

A  singular  root  rause  was  identified  generating 
multiple  TRRs; 

o  Failure  of  the  contractor  to  be  ready  tor 
test 

Multiple  TRRs  are  costly  to  both  the  Government 
and  contractor,  contribute  to  the  length  of  the  test 
schedule,  and  constitute  a  non-productive  expendi¬ 
ture  of  test  team  resources.  Data  indicates 
contractor  historically  is  not  ready  • 

Multiple  Government  TRRs  can  be  avoided  y 
placing  the  burden  of  test  readiness  solely  on  the 
contractor.  The  need  for  multiple  TRRs  should  be 
re-evaluated  and  contract  requirements  written  to 
reduce  the  number  of  TRRs  accordingly. 

Area  8:  CommSlSL.  PrOgzaiB-SlBigm 

Generation  ( CESGl  . .  r  j 

The  following  root  causes  were  identified  as  con¬ 
tributing  to  CPSG  requirements: 

o  Lack/weak  contractor  software  tools 
o  The  requirement  to  accommodate 
changes 

CPSG  requirements  constitute  a  lengthy  and  un¬ 
necessary  waste  of  resources  when  large  scale, 
complex  devices  are  involved.  Software  tools  and 
processes  are  available  which  provide  the  same 
level  of  confidence  in  the  integrity  of  the  software 
as  CPSG.  This  is  particularly  true  for  those  pro¬ 
grams  requiring  Ada  software  language  or  other¬ 
wise  require  automated  configuration  management 
routines.  Government  certification  of  contractor 
software  tool  and  processes  for  software  configura¬ 
tion  management  and  the  capability  to  support 
changes  should  be  accomplished  prior  to  test. 


Certification  would  then  allow  for  the  elimination 
of  CPSG  requirements. 


improvement 

The  CPT  is  recommending  fundamental  changes  to 
the  acceptance  test  process  and  activities  support¬ 
ing  that  process.  These  recommendations  are  based 
on  analysis  of  the  current  test  methodology  and 
solutions  formulated  by  the  team  to  eliminate  the 
root  causes  for  identified  wastes 
mended  new  process  known  as  Simulator  Test  20)0 
(ST  2000)  is  shown  in  Fig  2.  When  contrasted 
against  the  current  test  process  (ref  Fg  1),  the 
elimination  of  redundant  Government  CPSG  test¬ 
ing.  in-plant  performance  tests,  and  on-site  ac¬ 
ceptance  tests  becomes  obvious.  Accountabihty 
will  be  improved  by  better  aligning  authority  with 
responsibiUty  for  Government  and  contractor  de¬ 
velopment  test  teams.  Unnecessary  testing  will 
eliminated,  test  documentation  will  be  minimized, 
and  the  level  and  type  of  testing  will  be 
cused  on  satisfying  training  requirements.  In  addi¬ 
tion,  a  comprehensive  and  more  effective  “• 
sessment  will  be  realized  without  extensive  TRRs. 
Test  procedures  will  be  elevated  to  the  functional 
level  and  Air  Force  Subject  Matter  Experts 
(SMEs)  will  be  made  available  to  assist  the  Mn- 
tractor  during  systems  development.  Test  function¬ 
ality  will  not  be  reduced  nor  will  test  integrity  be 
compromised  as  a  result  of  the  recommendations 
proposed  in  ST  2(M)0. 

The  CPT  concentrated  on  the  task  of  improving  the 
efficiency  of  acceptance  testing.  However,  "testing 
encompasses  and  is  influenced  by  a  much  broader 
span  of  activity  outside  the  formal  test  program. 
Program  development  tasks  prior  to  acceptance 
testing  were  suspected  by  the  CPT  membership  of 
mas^ng  problems  which  subsequently  appear  dur¬ 
ing  or  delay  the  start  of  acceptance  testing.  The 


CPT  findings  show  a  major  cause  for  delay  in 
fielding  acceptable  training  devices  is  due  to  activity 
that  precedes  the  start  of  acceptance  testing.  In 
particular,  recommendations  made  m  the  ar^  of 
design  data,  aircraft  components,  and  Hard¬ 
ware/Software  Integration  are  emphasized  because 
of  their  known  historical  impact  on  the  test  pr^ 
gram.  Correction  of  these  problems  will  largely 
avoid  significant  delays  experienced  on  past  pro¬ 
grams. 
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With  regards  to  deagn  data,  the  CPT  recom¬ 
mended  that  SMEs  be  made  available  to  the  try¬ 
ing  device  contractor  early  in  the  program  to  help 
resolve  data  deficiency  problems  and  ^blish  con¬ 
sensus  on  interpretation  and  application.  Further, 
that  the  Government  implement  the  Simulator 
Data  Integrity  Program  study  recommendations  to 
ensure  timely,  accurate,  and  complete  daU  av^- 
ability  to  the  training  dewce  developer  from  the 
weapons  system  de^  contractor.  Also  recom¬ 
mended  is  the  use  of  more  aggressive,  rompre- 
hensive  performance  warranties  to  crystalia  con¬ 
tractor  liability  and  bolster  Government  confidence 
in  contractor  assertions  to  "meet  the  spedficauon. 
This  is  hoped  to  radically  reduce  conuact  test  re¬ 
quirements.  Lastly,  a  weU  defined  “d 
Training  Systems  Requirements  Analysis  should  be 
developed  prior  to  release  of  the  RFP  to  establish  a 
bound  training  tasks  early  in  program  development. 


To  relieve  the  schedule  delays  attributable  to  un¬ 
available  or  late  airaaft  components,  early  idcntiU- 
cation  of  requirements  (including  spares)  followed 
by  obtaining  sufficient  priority  for  timely  acquisi¬ 
tion  from  the  vreapons  system  contrartor  is  consid¬ 


ered  essential.  Components  could  manufac¬ 
tured  or  alternately  provided  by  the  trainmg  system 
developer  via  an  associate  contract  agreement  with 
the  weapons  system  developer.  Prime  contractors 
are  encouraged  to  strengthen  subcontrac¬ 
tor/vendor  management  processes  to  improve  de- 
Uvery  performance  and  reduce  the  impact  on  de¬ 
vice  readiness.  A  repair  pipeline,  negouated  with 
the  original  equipment  manufacturer  prior  to  tej- 
ing,  may  insure  the  availabiUty  of  spares  dunng  the 
test  program. 

Significant  recommendations  to  improve  hard¬ 
ware/software  integration  include  establishing  m 
Industry  lead  CPT  to  investigate  viable  HSl 
in-process  measurement  criteria.  Improved  man¬ 
agement  of  this  important  development  step  is 
critical  to  Contractor  Engineering  Verification 
Tests  (CEVT).  To  mitigate  technical  performance 
and  schedule  risks  use  of  prototype  testing  to  ma¬ 
ture  new  technology  appUcations  prior  to  at¬ 
tempted  insertion  into  Hardware/Software  Integra¬ 
tion  is  proposed.  To  abate  the  impact  of  t^ 
interruptions,  risk  management  pro^ams  must  be 
developed  and  implemented  to  anticipate  and  man- 
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age  possible  causes.  Training  device  development 
programs  requiring  new/modified  facilities  should 
consider  tasking  the  development  contractor  as  au¬ 
thorized  by  10  use  2353  to  centralize  contract  en- 
gineering  responsibilities  to  insure  facility  avail¬ 
ability. 

The  CPT  recognized  that  the  contractor  currently 
has  every  incentive  to  start  Government  test  to  see 
if  he  can  "sellofT  the  device  and  save  schedule.  If 
testing  fails  to  achieve  the  desired  result,  the  con¬ 
tractor  may  find  it  more  economical  to  resist  cor¬ 
rections,  attempt  short  term  solutions,  and  hope 
test  schedule  concerns  will  cause  the  Government 
to  weaken  its  position. 

However,  the  CPT  also  believes  that  exceptional 
contractor  test  performance  should  be  rewarded. 
Conduct  of  an  effective,  well  planned  test  program 
is  a  worthwhile  objective.  The  creation  of  contract 
incentives  to  accomplish  this,  however,  is  depen¬ 
dent  upon  several  variables  including  the  basic  na¬ 
ture  of  the  testing  (development  vs.  production), 
the  type  of  contract  (cost  vs.  fixed  price),  and  the 
type  of  incentive  (i.e.,  objective  performance  incen¬ 
tive  versus  subjective  award  fee  incentive).  These 
factors,  along  with  other  pertinent  facts,  must  be 
weighed  when  trying  to  assess  the  ability  to  create  a 
"real”  contract  motivator. 

Development  Testing  -  Cost  Type  Contract  - 
Award  Fee;  Because  of  the  very  nature  of  a  devel¬ 
opment  program,  it  would  be  extremely  difficult  to 
structure  a  performance  incentive  which  would  be 
meaningful.  Development  testing  by  its  very  nature 
is  intended  to  surface  problems  before  the  design  is 
frozen  and  moved  into  production.  The  key  is  solid 
test  planning  and  analysis  in  order  to  minimize  sur¬ 
prises.  These  areas  tend  to  be  quite  subjective 
when  attempting  to  establish  measurement  criteria. 
Award  fee  provisions  would  allow  tailoring  of  the 
incentive  from  period  to  period  as  the  program 
progresses  and  provide  multiple  opportunities  to 
reward  performance. 

Production  Acceptance  Testing  -  Fixed  Price  Type 
Contract  -  Performance  Incentive:  In  a  production 
environment,  test  requirements  are  usually  quan¬ 
tifiable.  Because  of  this,  the  Government’s  ability 
to  write  a  meaningful  incentive  at  the  time  of  con¬ 
tract  award  is  much  greater.  A  concern  remains 
that  the  incentive  being  structured  is  sufficient  in 
terms  of  dollars  or  corporate  visibility  to  be  an  ef¬ 
fective  motivator  and  is  in  balance  with  the  re¬ 
mainder  of  the  program.  From  a  cash  flow  or  liq¬ 
uidation  standpoint,  the  contractor  is  still  motivated 
to  push  for  contract  buy  off  in  order  to  claim 


progress  payments  and  profit.  Additionally,  the  in¬ 
centive  must  be  justifiable  in  terms  of  overall  sav¬ 
ings  to  the  Government  (i.e.,  reduced  Temporary 
Duty,  more  efficient  use  of  personnel,  etc.). 

The  CPT  recommended  that  initiatives  begun  by 
The  Training  Systems  SPO  on  the  Simulator  for 
Electronic  Training  (SECT)  and  Special  Opera¬ 
tions  Forces  Aircrew  Training  Systems  Programs 
(award  fee)  and  the  Digital  Radar  Land  Mass  Sys¬ 
tem  (performance  incentive)  under  development 
for  the  F-16C/D  weapon  system  trainer  be 
continued  and  applied  to  all  new  acquisition  pro¬ 
grams.  The  need  to  monitor  results  of  programs 
where  incentives  have  been  applied  is  considered 
extremely  important  to  ensure  the  level  of  value  is 
worthwhile  to  the  contractor,  that  the  incentives 
remain  realistic  and  achievable,  and  experien^ 
gained  through  their  application  is  reinvested  in 
future  programs. 

CONCLUSIONS 

The  premier  improvement  to  simulator  and  main¬ 
tenance  trainer  test  process  has  been  identified  as 
Simulator  Test  2000.  Reforms  to  reduce  Govern¬ 
ment  test,  strengthen  and  reallocate  contractor  test 
responsibilities,  refine  test  documentation,  and  dis¬ 
crepancy  management  encompasses  major  CPT 
recommendations. 

For  any  recommended  process  change  to  be  con¬ 
sidered  for  implementation,  a  measurable  im¬ 
provement  must  be  expected.  If  there  is  a  signifi¬ 
cant  anticipated  benefit  as  a  result  of  the  new  pro¬ 
cess,  a  high  degree  of  motivation  to  adopt  the  new 
process  will  be  present. 

A  comparison  of  the  current  simulator  test  ap¬ 
proach  shown  in  Fig  1  and  referred  to  here  as  the 
"Idealized  Weapon  System  Trainer  (WST)  Test 
Program",  can  be  made  to  the  ST  2000  process. 
The  Idealized  WST  Test  Program  assumes  that 
once  testing  begins,  it  progresses  and  is  completed 
without  delays  or  interruptions.  This  test  program 
includes  CVT,  Performance  and  Acceptance  Tests, 
and  in-plant  as  well  as  on-site  Operational 
Evaluations  as  depicted  in  Table  1.  The  total  test 
effort  of  forty-eight  (48)  weeks  required  by  the 
idealized  WST  test  program  can  then  be  compared 
to  estimates  for  a  WST  using  the  ST  2000  process 
as  shown  in  Table  2.  It  can  be  seen  that  the 
reduction  of  the  total  test  effort  from  forty-eight 
(48)  weeks  to  thirty  and  one  half  (30.5)  weeks 
represents  a  savings  of  approximately  thirty-seven 
(37)  percent. 
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Table  1.  Idealized  WSTTest  Program 


IN-PLANT 

ON-SITE 

TESTS; 

eVT 

Performance 

Test 

Ops 

Eval 

Acceptance  Ops 

Test  Eval 

TOTAL 

WEEKS; 

20 

18 

2 

6  2 

48 

Table  2. 

ST  2000  Process 

IN-PLANT 

ON-SITE 

TESTS; 

System 

Assessment 

CEVT 

SPADE* 

Functional  & 
Mission  Tests 

TOTAL 

WEEKS; 

1.5 

20 

6 

3 

30.5 

*  SPADE  =  System  Performance  and  Development  Evaluation 


The  consensus  of  the  CPT  members  is  that  with  the 
adoption  and  implementation  of  the  ST  2000  pro¬ 
cess,  a  significant  savings  in  resources  can  be  real¬ 
istically  expected  if  all  elements  of  ST  2000  process 
are  implemented. 

Additionally,  if  the  other  recommendations  and 
suggested  changes  are  also  implemented,  increased 
efficiency  in  training  device  acquisition  programs 
pan  be  achieved.  These  are  outside  the  formal  test 
phase,  but  directly  affect  the  start  or  progress  of 
tpcfing  The  potential  savings  to  cost/schedule 
which  fan  be  achieved  by  implementing  these 
rhangp-.;  are  not  to  be  overlooked.  The  largest 
waste  in  most  military  training  device  development 
programs  occurs,  for  a  variety  of  reasons,  prior  to 
the  device  being  ready  for  testing.  Data  collected 
showed  that  on  average  military  programs  are  132 
days  (26.5  work  weeks)  late  prior  to  bepnning  of 
test.  The  elimination  of  this  waste  would  result  in 
cost  and  schedule  overrun  savings  of  approximately 
eighteen  (18)  percent  over  the  life  of  a  planned 
thirty-six  (36)  month  program. 

The  Simulator  Test  2000  concept  has  been  em¬ 
braced  by  the  Training  Systems  SPO  and  imple¬ 
mented  on  new  program  starts.  The  Simulator  for 
Electronic  Combat  Training  Development 
Program,  the  Leading  Air  Trmning  Command 
Electronic  Warfare  Officer  Training  Initiative,  and 


the  Euro-Nato  Procedures  Trainer  Modernization 
Program  are  both  structured  to  take  full  advantage 
of  the  ST  2000  process. 

The  development  requiremenu  for  aircrew  train¬ 
ing  devices  for  the  Joint  Surveillance  Target  Attack 
Radar  System  will  similarly  adopt  the  ST  2000  im¬ 
provements  and  an  on-going  contract  for  C-17 
maintenance  training  devices  is  being  modified  to 
take  advantage  of  test  cycle  process  improvements 
and  reduce  government  test  program  requirements 
(man-months)  by  an  estimated  50  percent. 

The  most  significant  process  change  is  the  cus¬ 
tomer’s  agreement  to  accept  CEVT  test  results. 
Repetition  of  these  types  of  specification  compli¬ 
ance  tests  by  the  Government  are  no  longer  a  re¬ 
quirement.  This  commitment  eliminates  the  single 
largest  cycle  of  customer  testing  from  the  current 
acceptance  test  process. 

In  point  of  fact,  ST  2000  places  the  responsibility  to 
thoroughly  execute  CEVT  squarely  on  the  contrac¬ 
tor.  It  must  be  conducted  to  the  same  level  as  re¬ 
quired  for  developmental  performance  testing  pre¬ 
viously  conducted  by  the  Government.  This  means 
that  test  results  must  be  documented,  verifiable  and 
repeatable.  Failure  to  execute  stringent,  valid  test¬ 
ing  with  documented  results  will  motivate  the  cus¬ 
tomer  to  demand  a  repeat  of  previously  run  tests 
and  will  again  require  100%  witnessing  of  CEVT. 
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If  the  contractors  do  not  perform  their  part,  the 
customer  has  no  alternative  but  to  revert  back  to 
the  existing  test  philosophy.  The  customer  has  ex¬ 
tended  the  opportunity,  the  contractor  must  aggres¬ 
sively  resfxjnd  for  ST  2(XK)  to  be  viable. 
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COMBAT  PERFORMANCE  TRAINING  and  PIPELINE  CONNECTIVITY 

The  ASW  Aircrew  Model 
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ABSTRACT 

The  Anti-Submarine  Warfare  Sensor  Operator  (AW)  in  the  Navy’s  VS,  VP,  HS  and  HSL 
communities,  performs  a  function  which  is  not  only  mission-critical,  but  one  in  which  he  alone  has  been 
trained.  Solely  responsible  for  the  detection  and  classification  of  subsurface  contacts,  the  skill,  judgement 
and  training  of  this  individual  is  the  cornerstone  of  an  ASW  prosecution. 

The  AW  training  "pipeline"  reflects  the  traditional  military  approach  to  training,  which  emphasizes 
task-driven  procedural  instruction  in  the  operation  of  systems  and  equipment.  This  approach  neglects  the 
development  of  higher  order  cognitive  skills  necessary  to  the  creation  of  expert  operational  performers. 
This  forces  individual  squadrons  to  use  On-the-Job  Training  (OJT)  to  make  up  for  this  shortfall.  OJT 
however,  is  largely  an  ad  hoc  arrangement,  only  as  good  as  the  training  credentials  of  individuals  within 
a  given  command,  and  conducted  on  a  not-to-interfere-with  basis  with  preparations  for  the  next  readiness 
assessment  hurdle. 

The  Navy  cannot  afford  to  train  its  operators  and  tacticians  of  the  21st  Century  in  this  traditional 
manner.  Inefficiencies  in  current  methods  will  be  magnified  in  the  future,  and  could  inhibit  emerging 
technological  advantages.  A  shrinking  budget,  cutback  in  forces,  reduced  opportunities  to  practice,  and 
growing  complexity  of  the  threat  taxonomy,  all  call  for  a  complete  overhaul  in  the  way  the  Navy  trains 
and  supports  this  critical  mission  specialty. 

INTRODUCTION 

7r  cannot  be  too  often  repeated  that  in  modern 
war,  and  especially  in  modern  Naval  war,  the 
chief  factor  in  achieving  triumph  is  what  has 
been  done  in  the  way  of  thorough  preparation 
and  training  before  the  beginning  of  the  war.  " 

-  Theodore  Roosevelt:  Graduation  address, 

U.S.  Naval  Academy,  June  1902 

The  notion  that  there  isn’t  going  to  be  time 
to  get  ready  once  a  conflict  begins,  presents  a 
sound  argument  for  manning  military  forces  with 
personnel  who  not  only  have  the  basic 
procedural  skills  required  to  operate  equipment, 
but  also  the  tactical  decision-making  skills 
required  in  combat.  Given  that  a  trainee  is 
transformed  into  a  fleet  operator  on  graduation 
day,  the  Navy’s  training  pipeline  is  presented 
with  the  clear  challenge  of  developing  fleet 
operators  with  combat  performance  and  tactical 


decision-making  skills.  The  objectives, 
methodologies  and  evaluation  criteria  between 
procedural  and  performance  training  are 
significantly  different. 

This  paper  discusses  a  few  of  the 
preliminary  conclusions  from  an  ongoing  study 
of  the  ASW  Aircrewman  training  pipeline.  The 
study’s  objective  is  to  assess  current  training 
methodologies  in  order  to  identify  wasteful, 
duplicative,  unnecessary,  ineffective  or 
outmoded  methods.  The  ultimate  goal  is  to 
develop  a  continuous  training  system,  which 
creates  and  sustains  a  high  percentage  of  expert 
performers,  capable  of  meeting  the  threat 
environment  of  the  21st  Century. 

A  conclusive  assessment  of  any  training 
pipeline  must  determine  whether  there  is 
measurable  evidence  of  the  value  of  the  current 
system,  and  whether  previously  established  goals 
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and  training  methods  still  continue  to  be 
important.  Military  planners  need  to 
approximate  the  missions,  operating 
environment,  threat  capabilities,  and  operational 
weapons/sensor  systems  of  the  future  before  an 
elfective  training  process  can  be  defined.  This 
paper  attempts  to  "bound  the  problem"  by 
offering  preliminary  conclusions  and 
recommendations  with  respect  to  the  following 
are^:  Performance  Drivers,  Personnel 

Screening  and  Selection,  Training  Pipeline 
Connectivity,  Procedural  Training  Dominance, 
Mission  Planning,  Post-pipeline  training. 
Performance  Feedback,  and  Archives 
Development. 

THE  TRADITIONAL  APPROACH 
The  Navy’s  current  training  development 
strategy  was  defined  to  a  large  extent  by  the 
proliferation  of  the  Instructional  System 
Development  (ISD)  methodology  of  the  1960s, 
embraced  by  Naval  Aviation  in  the  1970s.  This 
approach  has  been  used  successfully  in 
maintenance  training,  where  economies  have 
been  realize  through  the  establishment  of  rigid 
standardized  procedures. 

The  ISD  approach  was  built  on  the  premise  that 
an  individual’s  performance  would  largely  be 
proc^ural  in  nature,  (e.g.  the  maintenance  and 
operation  of  systems  and  equipment),  and  that  an 
appropriate  program  of  training  could  be 
developed  by  first  identifying  and  segmenting 
these  procedural  duties,  and  then  developing 
specific  training  for  each  independent  tasks  and 
sub-task. 

The  key  strength  of  this  approach  is  its 
simplicity.  A  "task  listing"  provides  certainty 
when  actual  performance  requirements  may  be 
ambiguous  or  difficult  to  describe.  Simplicity 
however,  is  also  this  system’s  biggest  detractor. 
The  system  fails  when  it  is  relied  upon  to  teach 
higher  order  cognitive  processes  associated  with 
tactical  decision-making.  Simulated  ASW 
missions,  for  example,  conducted  in  Weapons 
Systems  Trainers  (WSTs)  and  on  instrumented 


ranges,  found  that  pipeline  graduates  -ave 
difficulty  visualizing  tactical  situatums, 
prioritizing  objectives,  handling  complex 
decision-making,  coordinating  with  other  crew 
members,  and  projecting  threat  actions.  Each  of 
these  skills  are  critical  to  performance  in 
combat.  Moreover,  while  an  acoustic  operator 
must  understand  oceanographic  theory  as  well  as 
the  procedural  tasks  associated  with  the 
operation  of  his  equipment,  what  separates  the 
novice  operator  from  the  "expert  performer"  are 
a  largely  undefined,  but  clearly  evident  group  of 
advanced  level  cognitive  skills,  that  the 
traditional  training  system  does  not  recognize,  or 
routinely  develop  in  its  graduates. 

Determining  the  cognitive  skill  requirements 
for  fleet  operators  is  more  difficult  than 
identifying  the  expert  performers  from  their 
counterparts.  Everyone  in  a  given  command 
usually  knows  who  the  operational  standouts  are. 
It’s  taken  for  granted  that  these  individuals  have 
mastered  the  procedural  aspects  of  their  jobs.  It 
is  during  demanding  tactical  or  emergency 
situations,  however,  that  the  superior 
performance  of  these  individuals  is  most 
conclusive.  When  faced  with  a  crisis,  they  shift 
into  "auto-pilot",  screening  and  prioritizing 
information,  processing  relevant  details, 
handling  complex  tasks  without  stopping  to  think 
about  individual  steps,  taking  decisive  action  at 
a  pivotal  moment  and  developing  contingency 
plans  for  anticipated  conditions.  It  is  every 
Commanding  Officer’s  goal  to  saturate  his 
command  with  talent  like  this,  yet  while 
individual  ships,  submarines  and  squadrons 
usually  have  a  number  of  these  individuals  at 
their  disposal,  it  is  usually  the  result  of  On-the- 
Job  Training  (OJT)  or  (personnel)  detailing. 
The  problem  with  relying  on  either  of  these 
sources  is  that  OJT  has  wide  variations  in 
quality,  frequency,  standardization,  and  priority 
between  commands,  and  personnel  detailing 
doesn’t  prepare  operators,  it  merely  shifts  talent 
between  commands.  The  bottom  line  is  that  a 
training  pipeline  that  fails  to  recognize  the 
difference  between  performance  training  and 
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procedural  training,  creates  a  lower  aggregate 
number  of  "expert  performers",  and  consumes 
time  and  resources,  producing  graduates  at  a 
higher  training  cost  than  is  necessary. 


Key  questions  which  the  current  training 
pipeline  haven’t  answered  are;  What  are 
true  performance-related  skill  requirements  for 
a  successful  fleet  operator?  How  does  one  go 
about  developing  a  training  strategy  w  ic 
stresses  the  cognitive  skills  critical  to  combat. 


PERFORMANCE  "DRIVERS" 
Essentially,  two  key  factors  determine  the 
potential  performance  capabilities  of  an 
individual:  innate  ability  (what  is  known  )  and 
p,Tcperience  .  (what  is  learned  ).  raining 
(iSSion,  practice  &  feedback)  is  the  process 
used  to  develop  experience.  Therefore,  creating 
a  system  which  is  capable  of  producing  expert 
performers  requires  an  appreciation  for  me 
forces  at  work,  specifically;  innam  ability, 
experience,  and  training.  Understanding  how  to 
isolate,  develop  and  nurture  each  of  *ese  is  a 
first  step  in  me  design  of  a  fonctional  model  for 

creating  "expert  performers". 


Innate  ability.  -  Intelligence,  motivation, 
attitude,  maturity,  perseverance,  concentration, 
logic,  and  analytical  proficiencies  all  play  strong 
roles  in  me  development  of  advanced  level 
cognitive  skills  so  often  found  in  expert 
performers.  Increasing  me  pool  of  personnel 
wim  me  right  mix  of  mese  talents  for  any 
specific  skill  area  are  recruiting,  screening  and 
personnel  management  (detailing)  issues,  which 
profoundly  affect  me  cost  of  operator  training  as 
well  as  me  level  of  expertise  in  critical  mission 
specialties  wimin  me  fleet. 


F.xnerience.  A  common  denominator  among 
many  "expert  performers"  is  me  rich  library  of 
previous  operational  experiences  mey  have 
internalized.  These  cognitive  blueprints  provide 
mem  a  reference  which  mey  are  able  to  access 
and  modify  when  participating  in  demanding 


tactical  scenarios.  Highly  experienced  operators 
wim  a  rich  experience  base  have  more 
streamlined  decision-making  skills,  increased 
ability  to  anticipate  mreat  actions,  better 
organization  and  retention  of  relev^t 
information,  and  enhanced  ability  to  handle 


Training.  Once  a  basic  foundation  has  been 
established  in  me  primary  skills,  m^ry  and 
procedural  tools  required  to  understand  how  to 
function  as  an  Aircrewman,  me  capstone  to  me 
process  of  creating  an  expert  performer  is 
development  of  advanced  level  cognitive  skills 
(e.g.  judgement,  tactical  decision-making,  et. 
al.).  Accepting  me  principle  mat  what  separates 
the  "expert  performers"  from  me  novices  is 
development  of  an  in-depm_ cognitive  library  of 
performance  experiences,  men  me  ultimate  goal 
of  an  effective  training  system  should  be  to 
systematically  expose  trainees  to  as  many  diverse 
and  operationally  significant  learning  scenarios 
as  possible.  This  would  include  bom  direct 
participation  in  a  variety  of  operationally 
realistic  practice  events,  and  me  study  of  lessons 
learned  from  archives  of  operational  exercises. 
Key  to  me  development  of  mis  kind  of  training 
approach,  are  me  following  elements; 

1)  clearly  defined  performance  objectives, 

2)  rigorous  (student)  mission  planning, 


3)  observation,  reconstruction  and 
analysis  by  qualified  "experts  . 

4)  immediate  diagnostic  fealback  and 
performance  accountability,  and 

5)  insertion  of  lessons  learned  into 

accessible  archives  for  future  reference. 


Given  mis  broad  outline  of  me  elements 
necessary  to  create  expert  performance,  let  us 
lake  me  current  AW  training  pipeline,  as  a 
representative  model  to  current  training 
philosophy,  and  determine  how  well  it  measures 
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Overview  ot  the  AW  training  pipeline,  rigure 
1 .  provides  an  overview  of  the  present  training 
pipeline  for  ASW  Aircrewman.  During  recruit 
training,  a  functional  skills  exam  ("ASVAB" 
test)  is  given  to  all  recruits.  The  score  on  this 
exam,  which  consists  of  basic  mathematics, 
general  science,  arithmetic,  mechanical  aptitude, 
etc...,  makes  a  recruit  eligible  for  various  job 
specialties  and  further  training.  Those 
designated  to  become  aircrewmen  are  sent  to  the 
Naval  Aircrew  Candidate  School  (NACCS),  in 
Pensacola  Florida.  Here  they  receive  additional 
general  military  instruction  along  with  aircrew- 
specific  training  in  swimming,  water  and  land 
survival,  general  aviation  fundamentals,  etc... 


ror  oasic  dcuu2»iiu  aiiu 
procedural  skills  training.  Finally,  they  are 
assigned  to  a  Fleet  Replacement  Squadron  where 
they  learn  how  to  operate  the  equipment  they 
will  be  using  in  the  fleet. 

Personnel  Screening  and  Selection:  Presently, 
the  only  selection  criteria  used  to  select  AWs 
from  the  general  population  of  new  recruits,  is 
a  minimum  established  score  on  selected 
portions  of  the  ASVAB  exam.  The  test  formula 
(GS  +  AR  4-  2MK),  which  takes  the  General 
Science  (GS),  Arithmetic  (AR)  and  Math 
Knowledge  (MK)  portions  of  the  ASVAB,  and 
was  designed  to  provide  a  predictor  of  the  ability 
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of  a  trainee  to  graduate  from  the  AW  "A" 
School,  and  was  not  designed  tp  identic 
individuals  with  (potentiah  performance  skills 
required  in  the  fleet.  If  the  skills  required  to 
graduate  from  AW  "A"  School  were  good 
predictors  of  fleet  performance,  then  this  would 
not  be  a  problem,  however  as  we’ll  see  in  a 
moment  -  this  is  not  the  case.  The  issue  is  that 
there  is  no  credible  model  of  expert 
performance  in  the  fleet,  from  which  one  can 
develop  a  profile  for  personnel  screening  and 
trainee  selection.  This  forces  the  Navy  to  start 
with  a  random  sample  of  individual  ability, 
motivation,  attitude  etc. . . .  Private  industry  has 
long  recognized  that  certain  personality  types 
perform  better  in  some  jobs  than  others.  The 
insurance  industry,  for  example,  screens 
applicants  for  sales  jobs  using  a  personality 
profile  of  its  top  salesman.  They  look  for 
specific  levels  of  confidence,  self-reliance, 
motivation,  perseverance,  level  of  nervous 
energy,  and  the  need  for  social  contact.  By 
establishing  a  template  of  the  personality  make¬ 
up  of  their  most  successful  performers,  they 
have  a  blue-print  for  hiring  potentially  successful 
personnel.  Ceruin  key  strengths  are  better 
suited  to  any  particular  job,  and  the  military  is 
no  exception. 

Performance  profiles  do  not  exist  for 
military  operators  in  the  field  of  tactical 
decision-making.  This  means  that  recruiting, 
selecting,  screening  and  detailing  decisions  are 
being  made  without  this  criteria  in  mind,  at  a 
predictably  higher  training  cost  than  would 
otherwise  be  necessary.  The  potential  exists  to 
lower  training  costs  by  identifying  those 
cognitive  skills  existing  in  expert  performers  by 
job  specialty,  and  building  a  screening 
mechanism  tailored  to  select  recruits  with  those 
abilities  and  motivations. 

Some  have  argued  that  increasing  die 
acceptance  threshold  on  the  already  existing 
ASVAB  test  would  generate  better  operators. 
They  point  to  statistics  in  pipeline  attrition  and 
re-training  (night  school)  as  the  basis  of  their 


recommendation.  However  this  alternative  has 
no  foundation.  Current  attrition  statistics  in  the 
pipeline,  (excluding  non-academic  items  such  as 
disciplinary  action,  et.  al.)  are  less  than  one 
percent.  The  explanation  for  this  low  rate, 
simply  put,  is  that  the  evaluation  criteria  and 
competence  naturally  sought  the  level  of 
training. 

Pipp.line  Connectivity. 

The  organizational  structure  of  this  pipeline 
shows  that  the  student  enters  and  exits  schools 
operating  under  four  separate  administrative 
Chains  of  Command,  (NAVSCHOLCOM, 
CNTECHTRA,  FASOTRAGU,  FUNCWINGS) 
with  various  charters,  mission  statements, 
curriculum  objectives,  etc....  Coordinating 
these  various  programs  of  training  is  difficult  for 
a  number  of  reasons,  not  the  least  of  which  is 
their  physical  separation,  and  the  very  different 
objectives  and  operating  procedures  of  the 
various  Administrative  Chains  of  Command. 
No  single  training  strategy  is  in  place  that 
governs  the  process,  and  while  the  participants 
attempt  to  coordinate  on  an  ad  hoc  basis,  no 
dedicated  pipeline  "model  manager  exists  to 
oversee  that  these  programs  are  mutually 
supportive. 

Attempting  to  implement  a  material  change 
throughout  the  training  pipeline  is  not  an 
insignificant  feat.  Each  of  the  administrative 
chains  of  command  have  their  own  operating 
environment,  change  procedures,  lines  of 
authority  and  communication,  funding  sources, 
and  perceived  responsibilities.  Furthermore, 
implementing  a  change  in  one  end  of  the 
pipeline,  mandates  that  the  other  players  change 
their  curriculum  too.  It’s  as  if  a  house  were 
being  built  by  four  different  architects,  each 
assigned  one  wall  and  a  portion  of  a  roof,  and 
each  with  a  vision  of  what  the  final  structure 
should  look  like.  Two  solutions  come  to  the 
forefront: 

(1)  Provide  responsibility  to  a  single 
architect,  for  the  entire  project;  or  (2)  Make 
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slements  (reter  lo  ngure  j-j. 

implementation  of  either  of  these  proposals 
would  mitigate  the  connectivity  issue,  yet  there 
are  tradeoffs  involved  with  each.  Solution  #1 
relies  on  a  single  entity  for  the  success  or  failure 
of  the  project.  This  has  advantages  associated 
with  control,  coupling  of  authority  and 
responsibility,  and  single-point  interaction  with 
the  end-user.  Solution  #2,  requires 
communication  among  the  participants  to  ensure 
continuity,  but  distributes  the  burden  of  training, 
and  may  decrease  the  chances  of  an  inherent  bsas 
of  a  single  entity. 

The  pipeline,  as  it  presently  exists,  is 
unnecessarily  complex  -  and  while  the  sheer 
number  of  trainees  it  produces  means  that  some 
of  them  are  being  adequately  trained  in  spite  of 
the  process,  the  Navy  can  no  longer  afford  the 
luxury  of  wasting  resources,  particularly  if  it  is 
going  to  operate  in  the  future  with  much  smaller 
forces  than  they  have  grown  accustomed  to. 
Either  a  single  training  facility  is  called  for,  or 
the  existing  training  facilities  must  develop  a 

training  a  cto  training  strategy. 


of  an  expert  AW,  (primary  skills,  theory, 
procedural  and  cognitive),  procedural  training 
dominates  the  training  pipeline,  beginning  with 
the  AW  A-School  through  formal  graduation 
from  each  individual  Fleet  Readiness  Squadron. 
At  the  A-School,  for  example,  the  principal 
curriculum  driver  is  the  14D1  trainer.  This 
device  is  used  extensively  to  teach,  practice  and 
evaluate  procedural  skills  related  to  the  operation 
of  this  equipment.  The  14D1  is  a  generic 
trainer  that  does  not  replicate  any  hardware  the 
trainees  will  see  or  use  in  the  fleet.  The 
student’s  ability  to  control  and  manipulate  the 
14D1  however,  accounts  for  over  60%  of  the 
final  grade  at  the  A-school,  and  ultimately  plays 
a  role  in  assignment  decisions  with  respect  to  the 
various  Air  ASW  communities.  While  the 
FASO  and  FRS  training  programs  do  have 
access  to  Part  Task  Trainers  (PTTs)  and  Weapon 
Systems  Trainers  (WSTs)  which  mirror  die 
equipment  used  in  the  fleet,  a  similar  situation 
exists  in  that  the  curriculum  is  almost 
exclusively  procedural  driven.  This  situation 
forces  fleet  squadrons  into  the  position  of 
developing  the  tactical  decision-making  skills 
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Figure  2.  Project  administration  example. 
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meet  this  obligation. 
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tactical  training  -  usually  designed  around  the 
Nava!  Flight  Officers.  While  this  may  sound 
like  valuable  training  for  AWs,  other  than 
designing  the  search  pattern,  the  Tactical 
Coordinator’s  job  doesn’t  really  begin  until  after 
the  AW  has  detectal  and  classified  the  target  -  a 
process  that  can  be  subverted  if  contact  is 
artificially  generated,  or  an  "easy"  target  is  used 
Just  to  get  the  tactical  problem  going.  These 
hurdles  have  to  be  understood  by  training 
developers,  so  that  they  can  ensure  that 
readiness  measurement  doesn’t  prevent  readiness 
development. 


In  preparation  for  the  battle  of  the  Nile,  in 
which  Admiral  Nelson  defeated  the  French  fleet 
at  anchor  under  the  command  of  Vice  Admiral 
Francois  Brueys  (I  Aug  1798),  Nelson  did 
something  that  Brueys  had  not:  he  had  spent 
months  at  sea  not  merely  drilling  his  force,  but 
discussing  and  planning  the  battle.  His  captains 
all  new  his  plans.  He  was  to  write  before  the 
battle  of  Trafalgar  that  "something  must  be  left 
to  chance",  but  nothing  was  left  undone  that 
could  be  foreseen.  [2]  General  Eisenhower,  in 
preparation  for  the  Normandy  invasion  was 
quoted  as  saying:  "...  planning  is  everything, 
the  plan  is  nothing."  Both  of  these  great 
military  leaders  understood  the  value  of  the 
planning  process.  It  forced  one  to  project  threat 
capabilities,  understand  their  intentions, 
anticipate  their  actions  and  develop 
contingencies.  Air  ASW  units  do  surprisingly 
little  mission  planning.  As  a  result,  these  skills 
are  not  emphasized  adequately  in  the  training 
pipeline.  During  actual  operational  exercises, 
mission  planning  is  often  confused  with  briefing, 
which  turns  into  the  passive  activity  of  "getting 
briefed".  Aircrews  rely  on  others  (ASWOCs 
and  ASWMODs)  to  develop  their  plans. 
Insufficient  time  is  taken  to  prepare  for  a 
mission.  A  checklist  mentality  has  replaced 
thorough  preparation  as  a  standard  practice  for 
getting  ready  for  an  ASW  exercise.  The 
pipeline  doesn’t  train  its  operators  in  the  skills  of 
mission  planning  because  many  of  its  instructors 


don’t  know  how  to  do  it.  It’s  a  lost  skill  that 
needs  to  be  rekindled,  because  it  is  a  key 
experience-builder  that  develops  cognitive 
blueprints,  which  ultimately  leads  to  expert 
performance. 


Pp.rfrtrmance  Feedback. 

Since  the  introduction  of  Operations 
Research  and  Operations  Analysis,  during  WW 
II,  the  U.S.  Navy  has  refined  and  relied  upon 
the  OR/OA  process  to  develop  tactics,  identify 
system  requirements,  and  approximate  conditions 
of  readiness.  In  a  large  sense,  these  skills  are 
vital  to  the  efficiency  and  capabilities  of  a 
military  force,  because  there  are  three  basic 
determinants  to  whether  a  weapon  will  land  on 
target:  Tactics  (Doctrine/Procedures),  Systems 
(Equipment/Hardware)  or  Training  (Personnel 
Capabilities).  These  three  constants  form  a  triad 
which  decide  the  outcome  of  battles,  oftentimes 
before  they  have  started. 

The  Tactical  Development  and  Evaluation 
(TAC  D&E)  process  relies  heavily  upon  OA 
principles  to  measure  and  evaluate  the 
effectiveness  of  proposed  or  existing  tactical 
doctrine.  Naval  Test  Centers  utilize  these 
methods  and  techniques  to  evaluate  systems  and 
equipment  to  be  introduced  into  the  fleet. 
Current  training  programs  however,  have  failed 
to  implement  this  methodology. 

The  building  blocks  for  this  analytical 
process  are  the  Measure  of  Effectivai^s 
(MOE)  and  Measures  of  Performance  (MOP). 
Effectiveness  has  to  do  with  results  of  given 
actions,  without  regard  for  the  methods  used  to 
gain  results.  An  action  is  effective  or  successful 
if  the  results  meet  established  objectives. 
Performance  has  to  do  with  the  knowledge, 
training  and  skill  of  personnel  and  the  methods, 
procedures  and  tactics  they  use  to  meet  an 
objective.  Performance  is  successful  if  the  best 
procedure  or  tactic  was  selected  and  executed 
skillfully  -  regardless  of  the  outcome.  Success 
is  in  making  the  best  choice. 
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operating  under  separate  administrative  chains  of 
command.  Theoretically,  the  AW  trainee 
sequentially  progresses  through  this  pipeline, 
building  upon  knowledge  learned  in  previous 
commands,  and  gaining  primary  skills, 
th^retical  knowledge,  procedural  tools  and 
cognitive  skills  along  the  way.  The  existing 
pipeline  however,  relies  almost  exclusively  upon 
traditional  methods  of  training,  emphasizing 
task-analysis  driven  curriculum  which  focuses  on 
procedural  skills. 

While  the  specific  factors  which  separate  the 
"expert  performers"  from  their  contemporaries 
need  to  be  definitized,  the  following 
characteristics  are  postulated;  increased 
exposure  to  complex  mission  scenarios  coupled 
with  expert  diagnostic  feedback,  immersion  in 
an  operationally  oriented  environment  that 
rewards  achievement,  sufficient  emphasis  and 
opportunities  to  plan  and  rehearse  mission 
scenarios,  and  access  to  lessons  learned  from 
other  exercise  events.  Implementing  these 
changes  will  take  an  agreed  upon  cradle-to-grave 
training  strategy,  and  a  single  training  model 
manager  who  has  detailed  knowledge  of  critical 
Measures  of  Performance,  and  the  authority  to 
reshape  existing  training  methodologies. 
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INTRODUCTION 


Baseline  research  was  conducted  as  early  as  1961  in  evaluating 
the  effectiveness  of  individualized  instructional  programs.  Among 
the  better-known  and  widely  implemented  systems  for  individualizing 
instruction  are:  Individually  Prescribed  Instruction  (IPI)  (Glaser, 
1968;  1970);  Individually  Guided  Education  (IGE)  (Klausmeier,  and 
others,  1977) ;  and  the  Utah  System  Approach  to  Individualized 
Learning  (U-SAIL)  (Hales,  1978) .  All  of  these  approaches  strive 
to  actively  involve  the  student  in  the  learning  process,  allow 
students  in  the  same  class  to  be  at  different  points  in  the 
curriculxim,  and  permit  the  teacher  to  give  more  individualized 
instruction. 

In  the  late  1960s  and  early  1970s,  the  National  Science 
Foundation  (NSF)  supported  the  development  and  demonstration  of  two 
major  computer-assisted  instruction  (CAI)  systems;  Programmed  Logic 
for  Automatic  Teaching  Operations  (PLATO)  (Murphy  and  Appel,  1977) 
and  Time-Shared,  Interactive,  Computer-Controlled  Information 
Television  (TICCIT)  (Alderman,  1978)  .  NSF  also  supported  a  large- 
scale  evaluation  of  these  systems,  carried  out  by  Educational 
Testing  Service  (ETS) .  This  is  the  largest  evaluation  of  CAI  that 
has  been  attempted,  so  it  has  become  a  major  piece  of  literature  in 
this  area. 

Conceptualization  of  the  two  computer-based  learning  systems 
was  quite  different.  PLATO  is  centralized;  until  recently  it  was 
operated  totally  from  the  University  of  Illinois.  TICCIT  is 
decentralized,  depending  only  on  the  local  computer.  PLATO  is 
designed  to  be  an  adjunct  to  classroom  instruction;  its  exercises, 
drills,  and  games  are  not  intended  to  be  a  full  course  of 
instruction.  TICCIT  is  designed  to  provide  all  of  the  instruction 
that  would  otherwise  be  given  through  classroom  lectures.  However, 
both  systems  do  seek  to  individualize  instruction  by  providing  a 
variety  of  materials  to  match  the  needs  of  the  student,  and  by 
offering  immediate,  nonjudgmental  feedback  on  performance. 


Since  the  early  days  of  computer-based  training  (CBT)  the 
accepted  method  of  implementation  has  been  that  of  presenting  e 
courseware  in  a  large  learning  center.  This  may  have  een  mo 
the  benefit  of  the  instructors  than  some  deficiency  in  the  CBT 
programs.  In  any  event,  this  presentation  strategy  has  bec^e  the 
norm  for  most  CBT  programs.  Multiple  student  stations, 
tied  together  over  a  local  area  network  (I^)  ,  using  a  file 

server  for  courseware  storage,  and  often  under  the  real  time 
control  of  an  instructor  using  a  dedicated  console  has  become  the 

acc@pt©d  standard « 

There  are  both  positive  and  negative  attributes  to  centralized 
CBT  training,  and  there  are  be  strong  advocates  in  both  camps.  In 
support  of  the  centralized  learning  center  concept  is  toe 
immediate  availability  of  subject  matter  experts  (SMEs)  and  ad^unc 
resources  to  resolve  unanticipated  student  difficulties  as  y 
arise.  In  essentially  real-time,  students  having  difficulties 
a  training  program  can  be  given  one-on-one  remedial 
extra  assignments  outside  the  curriculum,  or  personalized  ha 
exercises.  In  the  majority  of  cases,  this  attention  is  sufficient 
to  eliminate  the  noted  learning  difficulty » 

on  the  other  hand,  maintaining  a  centralized  training  facility 
for  a  widely  disbursed  workforce  is  expensive  and,  in  many  cases, 
not  very  effective.  Initial  training  is  adequately  satisfied  at 
the  training  command.  However,  toe  continuing  need 
training  or  upgrade  training  on  an  evolving  weapon  ^^^hem  re^ 
the  technicians  to  return  to  toe  centralized  training 
This  is  expensive  in  terms  of  both  expended  resources  and  reduce 
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The  implementation  of  transportable  CBT  easily  supports 
initial  training,  refresher  training,  upgrade  training,  proficiency 
maintenance,  and  tactics;  all  on  a  personal  computer  without 
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need  for  the  allocation  of  additional  training  resources.  This 
approach  equally  supports  operator,  maintenance,  and  team  training. 

(See  Figure  1) 
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1.  Transportable  Computer-Based  Training  (TCBT) 
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are  combined  they  provide  a  complete  self“paced  training  program 
that  satisfies  initial „  refresher,  and  upgrade  training 
requirements o  Separated,  these  components  stand  alone  as  modules 
for  review,  testing,  or  job  aids  long  after  the  initial  training  is 
completed o 


It  is  anticipated  that  training  might  be  initiated  in  one  of 
two  different  waysi  the  technicians  could  request  a  specific 
training  program  through  the  local  educational  officer  or  the 
weapon  system  provider  could  include  the  training  program  as  an 
integral  part  of  the  installation  or  up-grade  of  the  weapon  system. 
In  addition,  accuracy  could  be  maintained  between  the  required 
operator/maintainer  procedures  and  any  installed  operational  system 
through  the  distribution  of  concurrent  upgraded  training  programs. 

Training  packages  would  be  either  formal  or  informal.  Formal 
training  includes  those  programs  that  are  required  for  advancement, 
increased  levels  of  responsibility,  and  new  job  assignments.  In 
other  words,  successfully  completed  training  programs  would  be 
documented  in  the  trainee's  permanent  record.  Informal  training, 
on  the  other  hand,  is  that  training  which  supports  the  existing  job 
assignment  by  maintaining  or  increasing  the  individual's 
proficiency  level.  Examples  include  the  practice  of  a  skill  that 
must  be  performed  within  a  set  of  strict  critera,  the  performance 
of  an  infrequent  but  critical  skill,  or  reactions  to  emergency 
situations  that  are  too  dangerous  to  train  on  operational 
equipment . 
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Transportable  CBT  programs  are  given  to  the  trainees  as 
complete  training  packages  that  consist  of  all  presentation 
software,  courseware,  and  adjunct  materials.  Trainees  simply 
insert  the  training  diskettes  into  a  PC  during  inactive  periods  and 
progress  through  the  course.  Immediate  transition  from  the 
training  presentation  back  to  the  operational  environment  is 
accomplished  by  inserting  a  "bookmark"  and  exiting  the  program. 
Thus,  when  the  operator  returns  to  the  training  program  it  is 
possible  to  continue  from  the  previous  point  in  the  training 
process.  Some  major  benefits  of  this  approach  are:  training  can  be 
presented  in  small  increments  (i.e.,  1  hour  or  less  a  day  as  the 
operational  environment  permits)  without  job  interruption;  travel 
to  a  centralized  training  facility  is  reduced;  and  the  subject 
matter  may  be  reviewed  time  and  again  as  dictated  by  the 
operational  requirements. 


Throughout  the  presentation  of  the  training  courseware,  and 
completely  transparent  to  the  trainee,  performance  data  is 
captured.  Collected  data  may  include  course  information  (course 
number,  title,  version,  release  date) ,  trainee  information  (name, 
SSN,  duty  station,  work  assignment,  supervisor,  etc),  course  access 
(who  is  enrolled,  start  and  stop  dates,  how  often  they  access  the 
course,  etc),  time  (in  the  course  and  each  module,  lesson,  test), 
trainee  performance  (score  by  course,  module,  lesson,  correct  and 
incorrect  responses  for  each  test  question,  number  of  remediations, 
etc),  and  statistical  information  for  the  educational  officer, 
courseware  developers,  and  higher  levels  of  command.  When  the 
training  package  is  returned  to  the  originator,  this  performance 
data  is  easily  accessed  to  update  records  or  to  generate  hardcopy 

reports . 


430 


THE  FOTTOB  OF  TCBT 

The  future  applications  for  this  approach  are  unlinited=  The 
expansion  of  initial ^  refresher,  and  upgrade  training  applications 
are  certainly  viable  options,  but  what  about  sensor  operators 
receiving  updated  profiency  training  prior  to  deployment  (or 
enroute)  that  represent  specific  target  characteristics  or  tactics? 
Why  can't  this  same  approach  also  be  used  to  pass  technical  updates 
in  hardware,  software,  and  operating  procedures  to  the  fleet  either 
as  supplementary  materials  or  to  replace  hardcopy  manuals  and 
printouts? 

An  additional  application  might  be  termed  the  "what  if" 
scenarios o  These  would  be  fictitious,  but  realistic,  scenarios 
that  an  operator  might  face  in  the  operational  environment.  It 
could  include  a  saturated  air  picture,  complex  fleet  activity, 
tactics  simulations,  or  some  similar  event  that  would  reguire 
precision  responses  on  the  part  of  an  individual  operator  or  team. 
Maintenance  personnel  could  also  use  this  approach  on  PCs  to 
conduct  comprehensive  fault  isolation,  simulating  repair  and 
replace  procedures,  and  worJc^around  solutions.  The  application  is 
limited  only  by  the  imagination  of  the  courseware  developers  and 
the  ultimate  users. 


Ease  of  Update 


One  of  the  most  positive  attributes  of  the  TCBT  program  is  its 
ease  of  update  and  distribution.  Courseware  is  always  developed  as 
stand  alone  modules,  and  where  possible,  as  generic  modules  (for 
example,  basic  electronics  is  basic  electronics  regardless  of  the 
ultimate  application) .  Instructional  designers  can  then  mix  and 
match  existing  modules  with  new  modules  to  form  new  programs  or 
simply  add  the  upgrade  changes  to  reflect  the  update. 
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Distribution  is  easily  accomplished  by  sending  a  new  diskette 
to  each  addressee.  A  software  routine  will  remove  and  replace  the 
identified  module  or  lesson  with  the  new  version.  Hard  copy 
updates  are  accomplished  in  the  same  manner  using  the  appropriate 
word  processing  system. 

■Tust-in-Tlme  Training 

Just-in-time  training  refers  to  the  courseware  that  is 
resident  in  the  operational  system.  This  serves  as  a  readily 
available  performance  job  aid  when  needed  by  the  operator  « 
maintenance  technician.  This  has  been  implemented  very  effectively 
in  the  area  of  Automated  Test  Equipment  (ATE) . 

A  pull-down  menu  can  be  accessed  from  the  operational  screen 
in  which  specific  procedures  are  made  available.  This  may  include 
a  selection  of  schematics  in  which  the  technician  can  progressively 
zoom  and  pan  until  the  area  of  interest  is  centrally  located  on  the 
screen.  The  technician  may  then  move  about  the  drawing  and  make 
notes  as  desired.  Also  available  is  the  ability  to  call  a  variety 
of  digital  files  displaying  video  sequences  showing  the  operator 
how  to  perform  detailed  procedural  tasks,  for  example,  how  to 
adjust  one  of  many  potentiometers  until  the  waveform  shown  in  the 

video  clip  frame  is  obtained. 

Another  example  might  require  the  operator  to  PROBE  Rl,  PIN  1 
on  a  circuit  board.  The  board  might  be  shown  as  a  video  file  with 
the  specific  area  circled  and  the  probe  location  indicated  on  an 
enlarged  video  clip.  Without  this  approach,  the  operator  wou^ 
have  to  locate  a  drawing  in  a  Technical  Order,  then  find  the  part 
among  hundreds  of  callouts  on  the  page. 

CONCLUSIONS 

CBT  training  on  PCs  avoid  having  to  provide  instructors,  as 
well  as  reducing  the  cost  and  eliminating  the  inconvenience  of 
scheduling  operators  and  technicians  to  take  conventional  classroom 
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Total  Quality  Leadership  Awareness  Training 


by:  Wally  Lange 
Naval  Ocean  Systems  Center 
Associate  for  TQL 

There  is  a  new  leadership  concept  being  advocated  at  the  Naval  Ocean 
Systems  Center  known  as  Total  Quality  Leadership  or  TQL  for  short.  TQL  is 
both  a  philosophy  and  a  set  of  guiding  principles  that  represent  the 
foundation  of  a  CONTINUOUSLY  IMPROVING  organization.  TQL  is  the  application 
of  quantitative  methods  and  human  resources  to  improve  the  material  and 
services  supplied  to  an  organization,  all  the  PROCESSES  with  the  organization, 
and  the  degree  to  which  the  NEEDS  OF  THE  CUSTOMER  are  met,  now  and  in  the 
future.  TQL  integrates  fundamental  management  techniques,  existing 
improvement  efforts ,  and  technical  tools  under  a  disciplined  approach  focused 
on  CONTINUOUS  IMPROVEMENT. 

The  above  is  the  Department  of  Defense's  definition  of  TQL  and  is  the 
starting  point  for  all  DoD  activities  as  they  move  toward  implementing  TQL 
within  their  respective  commands.  At  the  base  of  the  structure  for 
implementing  TQL  at  any  command  is  the  critical  need  to  have  all  individuals 
have  an  understanding  and  working  knowledge  of  the  intent,  direction,  scope 
and  planned  results  of  the  TQL  effort.  At  the  Naval  Ocean  Systems  Center, 
Wally  Lange,  the  Center's  Associate  for  TQL,  has  developed  and  conducted  TQL 
Awareness  classes  for  the  past  year  and  a  half.  Over  650  Center  employees 
have  attended  classes  to  date.  The  reason  the  Naval  Ocean  Systems  Center  has 
taken  such  an  aggressive  effort  to  implement  TQL  can  be  clearly  seen  for  the 
the  following  reasons.  First,  TQL  has  the  highest  level  of  support  from  the 
Secretary  of  the  Navy,  the  Chief  of  Naval  Operations,  and  the  Commander,  Space 
and  Naval  Warfare  Systems  Command.  Secondly,  the  implementation  of  TQL 
requires  that  all  Center  employees  become  aware  of  its  principles,  goals,  and 
the  methods  by  which  the  Center  plans  on  implementing  TQL. 

Questions  that  this  paper  will  address  are  how  can  an  organization  conduct 
effective  TQL  Awareness  Training,  obtain  the  latest  thinking  on  the  subject 
material,  and  make  the  training  relevant  to  the  individuals  being  exposed  to 
the  TQL  philosophy. 

To  begin  with,  the  single  most  critical  element  of  the  training  effort  is 
to  have  the  full  support  and  enthusiastic  backing  of  the  activity  Commander  or 
Commanding  Officer  as  well  as  the  top  management  personnel  within  the 
organization.  When  the  Chief  of  Naval  Operations  says  to  audiences  that,  "He 
is  still  learning  about  TQL"  it  is  fairly  safe  to  say  that  all  others  within 
the  Navy,  military  and  civilian  alike,  should  have  the  same  viewpoint  of  their 
own  education  in  the  matter.  The  Navy  is  quickly  developing  standardized  and 
appropriate  level  training  courses  that  will  soon  be  available  to  all  Navy 
Commands.  Training  courses  are  also  available  now  from  the  Federal  Quality 
Institute,  that  is  part  of  the  Office  of  Personnel  Management.  However,  the 
problem  two  years  ago  was  that  these  above  mentioned  training  resources  were 
not  available  to  Department  of  Defense  activities. 


The  Naval  Ocean  Systems  Center  and  its  sister  Research,  Development,  Test 
and  Evaluation  laboratories  needed  to  institute  training  then  and  there  during 
the  1989-1990  period.  The  route  taken,  and  one  that  proved  for  the  Naval 
Ocean  Systems  Center  to  be  highly  effective,  was  that  a  self-developed  command 
Total  Quality  Awareness  Training  Course  was  produced.  It  was  produced  by 
having  a  few  individuals,  research  the  available  written  material,  share  ideas 
among  a  team  of  "TQL  Coordinators"  from  the  various  laboratories,  and  then 
develop  appropriate  outlines  and  viewgraphs  to  support  the  training  points. 

Two  major  documents  that  supported  the  TQL  doctrine  were  the  Total  Quality 
Management  Guide  -  Volume  I  (Key  Features  of  the  DoD  Implementation)  and 
Volume  II  -  (A  Guide  to  Implementation) ,  Published  by  the  Deputy  Assistant  of 
Defense  for  Total  Quality  Management,  these  documents  provided  the  best 
"current  thinking"  on  the  subject  upon  which  to  add  additional  relevant 
information.  The  most  significant  additional  information  was  from  the 
writings,  books  and  articles  by  the  master  of  quality  in  the  United  States, 

Dr.  W.  Edwards  Deming. 

The  next  step  was  the  need  to  identify  and  appoint  a  few  dedicated, 
knowledgeable,  motivated,  and  convincing  individuals  to  research  the  availa  e 
material,  distill  the  key  pearls  from  it.  and  develop  and  present  the  material 
in  an  interesting  and  enthusiastic  way  to  the  individuals  initially  interested 

in  learning  more  about  TQL. 

At  the  Naval  Ocean  Systems  Center,  it  was  top  management's  desire  to  have 
every  employee  hear  the  TQL  Awareness  training  message,  but  it  did  not  want  to 
mandLe  that  individuals  attend.  Better,  was  to  have  the  initial  recipients 
hear  the  message  and  then  by  word  of  mouth  and  verbal  praise  of  the  course 
convince  and/or  influence  others  to  attend.  Thus  the  "pebble  in  the  smooth 
pond"  theory  with  its  ever  spreading  areas  of  influence  has 
Lperience  at  the  Center.  To  date  over  650  individuals  have  participated  in 
the  TQL  Awareness  training,  about  20%  of  the  entire  Center  population  Also 
significant  is  that  approxiamtely  225  of  these  individuals  are  working  as 
members  of  the  Center's  Executive  Steering  Group,  six  functional  areas 
Management  Boards  and/or  20  Process  Action  Teams  and  are  actually 
principles  of  TQL  to  make  actual  changes  in  the  way  we  do  business.  This  the 
also  bLomes  a  "pebble  in  the  smooth  pond"  approach  to  having  more  and  more 
individuals  actually  involved  with  and  experienced  in  the  process  ot 
continuously  improving  everything  we  do. 

The  TQL  Awareness  training  should  be  of  an  overview  nature  and  not  so 
specific  as  to  bog  down  in  minor  details.  In  summary  the  two  sessions  of 
tKef  hours  each  are  taught  on  separate  days,  usually  in  the  morning  and 
address  the  following  subjects: 

Why  TQL  is  needed. 

15 -minute  video  by  the  CNO  on  why  it  is  important  to  him. 

TOL  Doctrine  -  (Short  overview  of  45  minutes) 

Discovering  the  Future:  The  Business  of  Paradigms.  Video  by  Joel  Barker 

Overview  of  Dr.  Deming' s  14  Management  Principles 

Tools  and  Techniques  used  to  collect  and  display  process  data. 

How  the  TQL  Infrastructure  (Executive  Steering  Group,  Quality 
Management  Boards,  and  Process  Action  Teams)  operates  with 
each  other  in  relation  to  the  existing  and  traditional  line 

management  organizations).  ,  rp 

Concludes  with  an  inspiring  and  informative  Video  by  Tom  Peters  - 

"A  Passion  for  Excellence" 
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io  fhe  training  must  be  conducted  in 

•  n  'au^  that  .llova  for 

relative  small  classes  of  „m-nrs  made  bv  the  instructor.  In  the 

an  open  exchange  and  discussion  o  because  of  the  class 

35  or  mort  tl-aes  conduc  ad  „«  tao^are  the 

participation.  What  we  a  y  g  something  is  fundamentally  different 

something,  but  to  get  them  einess  in  the  future.  The  major  difference 

about  how  we  are  going  to  do  our  business  in  the  future.  J 

is  that  we  will  have  top  Center  organizations .  both 

and  clearer  coimunications  ^ -^us"  the  classes  are  likewise  a  mixture  from  all 

tp.‘‘r“"’'aho“r«„nod.rfdl  rha  dlalo^e  aap.crs  of  rhe  TQh 

effort  can  be. 

Finally,  wa  anjoyabU.  ‘predb^rf^onur 

toward  improving  quality  can  and  will  y  ^  J  atmosphere  when  the 

initial  teaaa  a  high  laval  of  trust 

rng  re.7=  U%rd:ft  l^  t^eir^a^h  to  uaah  „aati„gs. 

adding  to  the  XQL  ~rFr‘,7^n"l  rnl 
:;:^°op:rin;‘:rorief  of  hlvt  h.d  dra.atlc  tum-rounds  and  outstanding 

results  when  the  principles  of  TQL  are  applie  . 

It  all  baglns  with  tha  first  stap  of  enployaa  TQL  awar.nass  training  and 
what  follows  will  be  amazing. 


Ab;u;  ;h;  Au*;!;  Wa'llV  langa  la  -  “->|X"l3of  ""'“h  “  2o''“yaar"mana'gaf " 
Center  and  is  the  Associate  for  Q  ,  .  ^  j  management  reviews,  manpower 

analyst/Naval  Rasarve  offloar  caraar  H.  r.ceivad 

dataVmlnatlon  studies,  and  the  davalopnant  “^'f®„"i“rslty.  For  fu. 

his  BA  dagraa  in  iTl-Alfo  ofAQTOVO^  553-0730. 

information,  please  call  Mr.  Lange  at 


For  further 
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